-- Logs begin at Thu 2022-09-22 02:05:47 UTC, end at Thu 2022-09-22 02:48:09 UTC. -- Sep 22 02:31:06.053155 dv-cinder-857075-7-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 22 02:31:06.061828 dv-cinder-857075-7-iscsi devstack@keystone.service[88051]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 22 02:31:06.063965 dv-cinder-857075-7-iscsi devstack@keystone.service[88051]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 22 02:31:06 2022] *** Sep 22 02:31:06.063965 dv-cinder-857075-7-iscsi devstack@keystone.service[88051]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 22 02:31:06.063965 dv-cinder-857075-7-iscsi devstack@keystone.service[88051]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 22 02:31:06.063965 dv-cinder-857075-7-iscsi devstack@keystone.service[88051]: nodename: dv-cinder-857075-7-iscsi Sep 22 02:31:06.063965 dv-cinder-857075-7-iscsi devstack@keystone.service[88051]: machine: x86_64 Sep 22 02:31:06.063965 dv-cinder-857075-7-iscsi devstack@keystone.service[88051]: clock source: unix Sep 22 02:31:06.063965 dv-cinder-857075-7-iscsi devstack@keystone.service[88051]: pcre jit disabled Sep 22 02:31:06.063965 dv-cinder-857075-7-iscsi devstack@keystone.service[88051]: detected number of CPU cores: 4 Sep 22 02:31:06.063965 dv-cinder-857075-7-iscsi devstack@keystone.service[88051]: current working directory: / Sep 22 02:31:06.063965 dv-cinder-857075-7-iscsi devstack@keystone.service[88051]: detected binary path: /usr/bin/uwsgi-core Sep 22 02:31:06.063965 dv-cinder-857075-7-iscsi devstack@keystone.service[88051]: your processes number limit is 47680 Sep 22 02:31:06.064577 dv-cinder-857075-7-iscsi devstack@keystone.service[88051]: your memory page size is 4096 bytes Sep 22 02:31:06.064577 dv-cinder-857075-7-iscsi devstack@keystone.service[88051]: detected max file descriptor number: 2048 Sep 22 02:31:06.064577 dv-cinder-857075-7-iscsi devstack@keystone.service[88051]: lock engine: pthread robust mutexes Sep 22 02:31:06.064577 dv-cinder-857075-7-iscsi devstack@keystone.service[88051]: thunder lock: enabled Sep 22 02:31:06.064577 dv-cinder-857075-7-iscsi devstack@keystone.service[88051]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 22 02:31:06.064577 dv-cinder-857075-7-iscsi devstack@keystone.service[88051]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 22 02:31:06.106314 dv-cinder-857075-7-iscsi devstack@keystone.service[88051]: Python main interpreter initialized at 0x55d11c9599c0 Sep 22 02:31:06.106314 dv-cinder-857075-7-iscsi devstack@keystone.service[88051]: python threads support enabled Sep 22 02:31:06.106314 dv-cinder-857075-7-iscsi devstack@keystone.service[88051]: your server socket listen backlog is limited to 100 connections Sep 22 02:31:06.106314 dv-cinder-857075-7-iscsi devstack@keystone.service[88051]: your mercy for graceful operations on workers is 90 seconds Sep 22 02:31:06.106685 dv-cinder-857075-7-iscsi devstack@keystone.service[88051]: mapped 403077 bytes (393 KB) for 2 cores Sep 22 02:31:06.106761 dv-cinder-857075-7-iscsi devstack@keystone.service[88051]: *** Operational MODE: preforking *** Sep 22 02:31:06.107046 dv-cinder-857075-7-iscsi devstack@keystone.service[88051]: *** uWSGI is running in multiple interpreter mode *** Sep 22 02:31:06.107046 dv-cinder-857075-7-iscsi devstack@keystone.service[88051]: spawned uWSGI master process (pid: 88051) Sep 22 02:31:06.107254 dv-cinder-857075-7-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 22 02:31:06.108364 dv-cinder-857075-7-iscsi devstack@keystone.service[88051]: spawned uWSGI worker 1 (pid: 88064, cores: 1) Sep 22 02:31:06.108364 dv-cinder-857075-7-iscsi devstack@keystone.service[88051]: spawned uWSGI worker 2 (pid: 88065, cores: 1) Sep 22 02:31:06.108364 dv-cinder-857075-7-iscsi devstack@keystone.service[88051]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 22 02:31:10.433242 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 22 02:31:10.433242 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: from cryptography.utils import int_from_bytes Sep 22 02:31:10.434078 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 22 02:31:10.434078 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: from cryptography.utils import int_from_bytes Sep 22 02:31:10.434531 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 22 02:31:10.434531 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: from cryptography.utils import int_from_bytes Sep 22 02:31:10.434639 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 22 02:31:10.434639 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: from cryptography.utils import int_from_bytes Sep 22 02:31:10.483537 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 22 02:31:10.483777 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 22 02:31:10.483933 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] command line args: [] {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 22 02:31:10.484025 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 22 02:31:10.484203 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 22 02:31:10.484288 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 22 02:31:10.484372 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ================================================================================ {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 22 02:31:10.484445 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] command line args: [] {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 22 02:31:10.484623 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] admin_token = **** {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.484709 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 22 02:31:10.484832 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ================================================================================ {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 22 02:31:10.484919 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] config_dir = [] {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.485045 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.485130 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] admin_token = **** {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.485229 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] config_source = [] {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.485314 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] config_dir = [] {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.485410 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] debug = True {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.485496 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.485613 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.485917 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] config_source = [] {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.485917 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] debug = True {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.486114 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.486219 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.486342 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.486439 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.486540 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.486651 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.486755 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.486889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.486999 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] list_limit = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.487102 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.487211 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] log_config_append = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.487306 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] list_limit = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.487412 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.487510 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] log_config_append = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.487605 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] log_dir = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.487700 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.487795 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] log_file = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.487896 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] log_dir = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.488009 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.488088 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] log_file = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.488199 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.488320 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.488402 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.488479 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.488604 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.488690 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.488763 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.488887 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.488971 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.489052 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.489175 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.489252 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.489368 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.489450 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.489534 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.489614 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.489739 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.489821 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.489903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.490038 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.490117 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.490279 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.490358 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.490517 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.490611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.490769 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.490890 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.491046 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.491325 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.491405 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.491527 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] publish_errors = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.491738 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.491823 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.491973 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.492056 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] publish_errors = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.492138 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.492215 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.492388 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.492475 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.492598 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.492696 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.492846 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] standard_threads = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.492930 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.493078 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.493156 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.493315 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] standard_threads = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.493398 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.493496 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.493573 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.493673 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] use_journal = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.493783 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.493931 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] use_json = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.494014 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.494086 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] use_stderr = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.494162 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] use_journal = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.494234 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] use_syslog = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.494310 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] use_json = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.494389 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.494462 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] use_stderr = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.494630 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] use_syslog = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.494713 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.494884 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 02:31:10.494976 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.495148 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.495227 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.495391 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.495467 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.495639 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.495716 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.495878 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.495958 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.496127 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.496205 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.496365 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.496443 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.496614 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.496700 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.496853 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.496930 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.497103 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.497216 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.497326 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.497407 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.497573 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.497652 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.497810 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.497887 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.498057 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.498142 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.498296 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.498384 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.498530 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.498607 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.498786 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.498889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.499054 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.499161 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.499332 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.499420 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.499570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.499683 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.499815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.499895 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.500062 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.500155 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.500320 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.500397 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.500570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.500654 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.500803 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.500889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.501037 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.501115 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.501269 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.501346 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.501505 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.501594 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.501743 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.501825 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.501985 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.502069 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.502216 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.502302 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.502448 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.502526 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.502695 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.502780 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.502951 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.503042 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.503194 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.503277 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.503443 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.503526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.503677 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.503762 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.503915 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.503994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.504151 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.504228 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.504381 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.504466 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.504613 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.504691 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.504846 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.504927 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.505078 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.505234 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.505310 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.505379 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.505534 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.505617 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.505769 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.505859 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.506007 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.506092 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.506238 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.506316 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.506477 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.506566 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.506710 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.506814 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.506969 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.507077 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.507207 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.507322 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.507447 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.507568 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.507687 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.507769 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.507928 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.508055 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.508166 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.508252 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.508406 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.508497 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.508636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.508715 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.508881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.509000 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.509117 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.509194 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.509356 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.509442 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.509591 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.509678 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.509832 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.509904 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.510072 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.510163 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.510304 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.510383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.510543 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.510622 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.510782 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.510876 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.511060 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.511140 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.511304 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.511384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.511555 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.511636 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.511804 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.511884 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.512042 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.512120 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.512279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.512365 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.512516 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.512596 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.512756 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.512839 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.512991 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.513069 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.513230 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.513311 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.513470 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.513553 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.513706 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.513784 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.513953 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.514037 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.514189 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.514266 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.514425 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.514580 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.514672 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.514747 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.515421 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.515504 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.515664 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.515742 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.515904 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.515985 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.516148 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.516227 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.516394 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.516471 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.516636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.516724 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.516873 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.516952 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.517118 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.517195 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.517342 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.517416 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.517578 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.517657 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.517814 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.517902 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.518047 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.518135 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.518278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.518358 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.518509 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.518593 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.518743 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.518850 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.519000 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.519078 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.519239 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.519316 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.519474 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.519563 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.519719 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.519797 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.519954 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.520031 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.520205 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.520286 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.520448 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.520536 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.520693 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.520776 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.520928 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] auth.external = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.521040 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.521161 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.521257 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.521425 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] auth.external = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.521508 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.521664 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.521751 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.521900 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] auth.password = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.521985 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.522132 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] auth.token = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.522212 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.522375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.522453 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] auth.password = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.522611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.522698 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] auth.token = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.522880 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.522966 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.523118 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.523197 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.523366 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.523455 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.523606 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.523686 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.523838 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.523916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.524082 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] credential.caching = True {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.524172 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.524322 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.524400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.524563 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] credential.caching = True {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.524642 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.524811 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.524894 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.525055 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.525135 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.525301 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.525380 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.525537 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.525624 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.525785 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.525864 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.526022 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.526099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.526265 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.526352 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.526519 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.526597 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.527040 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.527123 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.527302 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.527391 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.527568 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.527646 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.527813 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.527892 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.528070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.528157 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.528307 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] federation.caching = True {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.528386 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.528546 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.528626 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.528793 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] federation.caching = True {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.528878 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.529036 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.529115 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.529266 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.529343 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.529516 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.529603 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.529748 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.529826 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.529998 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.530078 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.530233 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.530312 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.530539 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.531205 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.531322 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.531402 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.531583 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.531666 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.531829 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.531914 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] identity.caching = True {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.532077 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.532155 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.532319 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] identity.caching = True {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.532400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.532554 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.532632 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.532791 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.532876 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.533040 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.533119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.533273 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.533350 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.533513 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.533592 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.533747 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.533834 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.533992 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.534071 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.534230 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.534298 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.534465 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.534546 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.534700 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.534780 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.534957 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.535036 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.535196 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.535280 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.535443 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.535521 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.535690 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.535768 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.535936 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.536016 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.536191 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.536270 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.536435 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.536513 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.536674 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.536755 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.536909 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.536989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.537149 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.537228 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.537382 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.537463 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.537620 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.537700 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.537854 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.537932 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.538093 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.538173 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.538334 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.538412 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.538565 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.538642 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.538816 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.538917 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.539084 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.539163 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.539319 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.539397 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.539569 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.539651 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.539808 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.539886 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.540043 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.540129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.540282 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.540352 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.540512 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.540586 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.540751 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.540842 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.540987 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.541069 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.541229 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.541313 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.541468 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.541554 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.541704 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.541782 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.541943 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.542021 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.542184 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.542314 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.542436 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.542551 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.542786 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.542922 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.543063 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.543156 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.543299 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.543377 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.543543 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.543621 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.543783 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.543871 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.544026 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.544105 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.544260 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.user = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.544338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.544499 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.544586 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.544737 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.user = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.544817 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.544980 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.545059 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.545234 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.545322 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.545470 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.545549 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.545719 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.545798 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.545957 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.546065 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.546218 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.546298 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.546458 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.546539 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.546691 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.546778 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.546953 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.547033 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.547188 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.547268 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.547429 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.547520 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.547665 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.547746 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.547911 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.547989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.548157 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.548248 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.548402 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.548481 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.548633 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.548712 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.548882 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.548970 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.549114 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.549193 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.549350 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.549428 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.549588 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.549676 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.549826 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.549906 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.550065 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.550143 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.550307 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.550399 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.550544 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.550622 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.550795 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.550902 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.551062 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.551155 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.551325 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.551406 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.551561 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.551641 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.551802 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.551888 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.552043 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.552122 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.552285 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.552364 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.552525 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.552612 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.552760 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.552840 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.552999 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.553085 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.553235 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.553322 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] resource.caching = True {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.553479 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.553559 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.553726 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] resource.caching = True {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.553805 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.553966 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.554052 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.554216 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.554296 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.554457 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.554535 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.554694 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.554781 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.554976 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.555057 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.555215 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.555294 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.555446 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.555524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.555681 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] role.caching = True {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.555762 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.555922 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.556007 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] role.driver = sql {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.556154 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.556234 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] role.caching = True {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.556402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] role.driver = sql {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.556481 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.556641 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.556721 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.556888 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.556968 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.557138 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.557217 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.557383 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.557462 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.557633 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.557713 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.557880 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.557959 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.558127 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.558208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.558358 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.558438 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.558607 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.558687 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.558890 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.558974 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.559115 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.559197 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.559361 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.559440 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.559610 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.559798 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.559886 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.560038 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.560277 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.560357 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.560519 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.560598 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.560765 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.560845 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.561016 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.561096 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.561252 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.561341 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.561510 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.561589 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.561776 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.561857 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.562019 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.562110 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.562254 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.562335 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.562526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.562611 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.562763 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.562871 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.563037 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.563117 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.563278 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.563360 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.563531 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.563618 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.563771 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.563852 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.564013 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.564093 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.564246 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.564335 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.564485 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] token.caching = True {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.564565 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.564719 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.564798 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.564961 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] token.caching = True {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.565048 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.565197 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.565276 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.565447 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.565525 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.565690 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.565777 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.565929 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.566007 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.566173 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.566254 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.566418 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.566506 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.566665 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.566745 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.566926 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.567010 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.567169 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.567261 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.567407 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.567486 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.567653 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.567733 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.567894 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.567982 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.568145 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.568225 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.568388 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.568476 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.568653 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.568740 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.568902 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.568984 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] database.connection = **** {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.569149 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.569230 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.569398 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.569487 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] database.connection = **** {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.569636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.569718 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.569879 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.569961 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.570116 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.570196 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.570354 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.570434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.570591 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.570671 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.570841 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.570924 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.571093 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.571171 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.571329 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.571410 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.571575 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.571655 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.571809 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.571888 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.572054 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.572134 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.572289 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.572368 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.572533 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.572613 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.572762 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.572845 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.573003 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.573083 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.573235 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.573313 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.573400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88065) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 22 02:31:10.573476 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.573707 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 02:31:10.573882 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88064) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 22 02:31:10.604383 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88065) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.604611 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.604700 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88065) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.604777 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.604852 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88065) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.604957 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.613556 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88065) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.613706 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.642436 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.642666 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.642987 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88065) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.643227 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88065) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.697371 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.697530 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88065) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.746803 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88065) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.747061 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88065) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.747148 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.747374 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.905857 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.906044 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88065) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.906158 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.906239 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88065) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.906336 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.906416 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88065) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.906499 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.906637 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88065) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.906720 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.906876 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88065) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.906963 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.907098 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88065) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.929838 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.930070 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.930191 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88065) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.930303 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.930503 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88065) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.930588 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.930718 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.930800 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88065) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.930971 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.931058 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88065) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.931189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.931273 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88065) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.931456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88065) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.931859 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88065) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.943228 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.945191 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88065) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.956160 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.956338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.956577 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.956804 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.957013 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88065) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.957109 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.957275 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.957378 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88065) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.957473 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88064) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.957553 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88065) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.957668 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88065) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.957875 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88065) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.958091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88065) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.958308 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88065) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 02:31:10.974622 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:10.974848 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 22 02:31:10.975202 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:10.975283 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:10.975413 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 22 02:31:10.975512 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 22 02:31:10.975694 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:10.975776 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:10.975962 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:10.976042 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 22 02:31:10.976225 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:10.976307 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:10.976485 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:10.976564 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:10.976746 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:10.977006 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:10.986656 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:10.986965 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:10.987159 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:10.987241 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:10.987441 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:10.987535 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:10.987715 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:10.988000 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:10.994466 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88064) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 02:31:10.994965 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88065) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 02:31:10.997412 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88064) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 02:31:10.997703 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:10.997850 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88065) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 02:31:10.997992 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:10.998138 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:10.998282 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:10.998425 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:10.998574 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:10.998714 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:10.998875 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:10.999047 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:10.999180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:10.999370 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:10.999489 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:10.999647 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:10.999766 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:10.999927 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.000045 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.000200 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.000319 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.000478 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.000750 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.023044 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.023695 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.024541 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88064) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 02:31:11.024763 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88065) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 02:31:11.024845 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.025051 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.029764 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88064) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 02:31:11.029958 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88065) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 02:31:11.030055 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.030249 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.030329 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.031082 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.037197 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88064) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 02:31:11.037388 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88065) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 02:31:11.037472 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.037677 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.043328 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88064) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 02:31:11.043515 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88065) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 02:31:11.043633 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.043817 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.043929 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.044108 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.044223 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.044399 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.044511 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.044686 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.044798 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.044971 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.045079 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.045248 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.045356 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.045521 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.061494 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.061627 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.061806 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.062014 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.065684 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88064) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 02:31:11.065856 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88065) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 02:31:11.068967 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.069098 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.069271 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.069398 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.073498 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88064) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 02:31:11.073638 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88065) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 02:31:11.076657 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88064) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 02:31:11.076801 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88065) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 02:31:11.079905 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.080042 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.080201 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.080327 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.080488 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.080613 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.080773 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.080899 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.081053 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.081186 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.081337 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.081463 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.097420 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88064) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 02:31:11.097707 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88065) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 02:31:11.097851 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.097977 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.098110 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.098234 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.098358 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.098514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.104997 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.105348 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.106528 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.106915 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.108079 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.108363 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.108491 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.109303 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.111730 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88064) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 02:31:11.112023 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.112150 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88065) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 02:31:11.112319 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.112446 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.112600 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.112726 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.112868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.112999 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.113268 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.124202 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88064) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 02:31:11.124386 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88065) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 02:31:11.124529 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.124771 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.124908 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.125033 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.125157 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.125361 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.125488 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.125700 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.125836 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.125994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.126122 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.126345 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.141419 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88064) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 02:31:11.141849 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88065) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 02:31:11.144373 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88064) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 02:31:11.144773 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88065) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 02:31:11.147314 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.147755 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.148903 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.149231 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.150384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88064) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 02:31:11.150673 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88065) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 02:31:11.150803 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.151006 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.151155 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.151378 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.158237 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.158500 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.159737 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88064) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 02:31:11.159990 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88065) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 02:31:11.162618 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.162876 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.163059 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.163279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.163361 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.163570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.163651 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.163852 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.172017 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88064) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 02:31:11.172225 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88065) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 02:31:11.172308 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.172528 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.172609 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.172812 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.179521 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88064) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 02:31:11.179656 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88065) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 02:31:11.179813 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.179939 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.180101 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.180228 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.180384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.180510 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.180669 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.180801 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.180949 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.181033 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.181228 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.181319 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.181528 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.181610 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.181802 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.181883 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.182080 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.182166 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.182366 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.182446 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.182640 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.182720 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.182944 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.183031 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.183231 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88064) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.183313 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88065) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 02:31:11.217558 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55d11c9599c0 pid: 88065 (default app) Sep 22 02:31:11.217757 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55d11c9599c0 pid: 88064 (default app) Sep 22 02:31:11.221446 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d9dd6-e562-423b-b7ba-03e4199b8531 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:11.221733 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d9dd6-e562-423b-b7ba-03e4199b8531 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:11.222006 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d9dd6-e562-423b-b7ba-03e4199b8531 None None] PATH_INFO: `/v3` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:11.226155 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 1/1] 10.222.0.20 () {50 vars in 791 bytes} [Thu Sep 22 02:31:11 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:18.134473 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66d5b4-f212-4011-baf5-25d558688885 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:18.135069 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66d5b4-f212-4011-baf5-25d558688885 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:18.135069 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66d5b4-f212-4011-baf5-25d558688885 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:18.140076 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 1/2] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:31:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:31:18.146378 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fdb6be-d8df-4835-8ce0-f9c33f30be26 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:18.147081 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fdb6be-d8df-4835-8ce0-f9c33f30be26 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:18.147081 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fdb6be-d8df-4835-8ce0-f9c33f30be26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:18.220125 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: INFO dbcounter [None req-17fdb6be-d8df-4835-8ce0-f9c33f30be26 None None] Registered counter for database keystone Sep 22 02:31:18.239327 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG oslo_db.sqlalchemy.engines [None req-17fdb6be-d8df-4835-8ce0-f9c33f30be26 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88065) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 22 02:31:18.245839 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writer thread running {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 22 02:31:18.435330 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG passlib.handlers.bcrypt [None req-17fdb6be-d8df-4835-8ce0-f9c33f30be26 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=88065) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 22 02:31:18.435715 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG passlib.handlers.bcrypt [None req-17fdb6be-d8df-4835-8ce0-f9c33f30be26 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88065) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 22 02:31:18.496287 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-17fdb6be-d8df-4835-8ce0-f9c33f30be26 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:31:18.497934 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-17fdb6be-d8df-4835-8ce0-f9c33f30be26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:31:18.627695 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 2/3] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:31:18 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 482 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 22 02:31:18.937576 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66d5b4-f212-4011-baf5-25d558688885 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:19.007486 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: INFO dbcounter [None req-ca66d5b4-f212-4011-baf5-25d558688885 None None] Registered counter for database keystone Sep 22 02:31:19.029743 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG oslo_db.sqlalchemy.engines [None req-ca66d5b4-f212-4011-baf5-25d558688885 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88064) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 22 02:31:19.044962 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writer thread running {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 22 02:31:19.191242 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66d5b4-f212-4011-baf5-25d558688885 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:19.195875 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb31d151-f22d-4b6e-a9e6-a5653c9c3e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f92fe1b089244bf4b21f4f98339aa81d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:19.196528 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31d151-f22d-4b6e-a9e6-a5653c9c3e19 admin admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:19.196820 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31d151-f22d-4b6e-a9e6-a5653c9c3e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:19.197113 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31d151-f22d-4b6e-a9e6-a5653c9c3e19 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:19.206079 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb31d151-f22d-4b6e-a9e6-a5653c9c3e19 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:19.258060 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: WARNING keystone.common.rbac_enforcer.enforcer [None req-fb31d151-f22d-4b6e-a9e6-a5653c9c3e19 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 22 02:31:19.258409 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb31d151-f22d-4b6e-a9e6-a5653c9c3e19 admin admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:19.269074 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 2/4] 10.222.0.20 () {58 vars in 1324 bytes} [Thu Sep 22 02:31:18 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 02:31:19.275636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fdb6be-d8df-4835-8ce0-f9c33f30be26 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:19.293571 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fdb6be-d8df-4835-8ce0-f9c33f30be26 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:19.298677 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217ccbf-9eef-402d-8586-6ac52c4ddc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f92fe1b089244bf4b21f4f98339aa81d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:19.299864 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217ccbf-9eef-402d-8586-6ac52c4ddc7e admin admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:19.300153 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217ccbf-9eef-402d-8586-6ac52c4ddc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:19.300431 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217ccbf-9eef-402d-8586-6ac52c4ddc7e admin admin] PATH_INFO: `/v3/projects/f92fe1b089244bf4b21f4f98339aa81d` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:19.302096 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a217ccbf-9eef-402d-8586-6ac52c4ddc7e admin admin] RBAC: Authorizing `identity:get_project(project_id=f92fe1b089244bf4b21f4f98339aa81d)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:19.354097 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a217ccbf-9eef-402d-8586-6ac52c4ddc7e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 22 02:31:19.354450 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a217ccbf-9eef-402d-8586-6ac52c4ddc7e admin admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:19.359770 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 3/5] 10.222.0.20 () {56 vars in 1207 bytes} [Thu Sep 22 02:31:19 2022] GET /identity/v3/projects/f92fe1b089244bf4b21f4f98339aa81d => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:21.647843 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f43f9b-e465-4d70-96bc-0fd52cc9d679 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:21.648432 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f43f9b-e465-4d70-96bc-0fd52cc9d679 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:21.648432 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f43f9b-e465-4d70-96bc-0fd52cc9d679 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:21.652844 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 3/6] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:31:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:31:21.659570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b535466-429c-4d00-9806-856f49db9de3 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:21.660331 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b535466-429c-4d00-9806-856f49db9de3 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:21.660479 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b535466-429c-4d00-9806-856f49db9de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:21.682211 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-2b535466-429c-4d00-9806-856f49db9de3 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:31:21.683866 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-2b535466-429c-4d00-9806-856f49db9de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:31:21.692107 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 4/7] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:31:21 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 22 02:31:21.967027 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f43f9b-e465-4d70-96bc-0fd52cc9d679 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:21.980059 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f43f9b-e465-4d70-96bc-0fd52cc9d679 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:21.984434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc7704-3027-4850-b39e-026cf5cb4690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f92fe1b089244bf4b21f4f98339aa81d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:21.985048 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc7704-3027-4850-b39e-026cf5cb4690 admin admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:21.985342 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc7704-3027-4850-b39e-026cf5cb4690 admin admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:21.985622 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc7704-3027-4850-b39e-026cf5cb4690 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:21.994085 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdc7704-3027-4850-b39e-026cf5cb4690 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:21.995936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdc7704-3027-4850-b39e-026cf5cb4690 admin admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:22.005886 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 4/8] 10.222.0.20 () {58 vars in 1324 bytes} [Thu Sep 22 02:31:21 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 02:31:22.012130 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b535466-429c-4d00-9806-856f49db9de3 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:22.025326 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b535466-429c-4d00-9806-856f49db9de3 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:22.030770 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ad855d-2557-48aa-b71f-e45d0339b02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f92fe1b089244bf4b21f4f98339aa81d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:22.031519 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad855d-2557-48aa-b71f-e45d0339b02f admin admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:22.031846 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad855d-2557-48aa-b71f-e45d0339b02f admin admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:22.032166 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad855d-2557-48aa-b71f-e45d0339b02f admin admin] PATH_INFO: `/v3/users/778bd65251774785bdd2460a6778044c` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:22.033582 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ad855d-2557-48aa-b71f-e45d0339b02f admin admin] RBAC: Authorizing `identity:get_user(user_id=778bd65251774785bdd2460a6778044c)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:22.035543 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ad855d-2557-48aa-b71f-e45d0339b02f admin admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:22.037203 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 5/9] 10.222.0.20 () {56 vars in 1198 bytes} [Thu Sep 22 02:31:22 2022] GET /identity/v3/users/778bd65251774785bdd2460a6778044c => generated 247 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:24.326878 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97021e83-d68d-41ee-a34c-8bbe3a797790 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:24.327788 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97021e83-d68d-41ee-a34c-8bbe3a797790 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:24.327788 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97021e83-d68d-41ee-a34c-8bbe3a797790 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:24.329174 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 5/10] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:31:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:31:24.336567 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448644c7-3269-4aa6-afd3-377a0e5f3aa5 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:24.337183 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448644c7-3269-4aa6-afd3-377a0e5f3aa5 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:24.337663 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448644c7-3269-4aa6-afd3-377a0e5f3aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:24.368785 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-448644c7-3269-4aa6-afd3-377a0e5f3aa5 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:31:24.370905 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-448644c7-3269-4aa6-afd3-377a0e5f3aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:31:24.454461 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d605506a-f8ba-4b6f-8d83-edbd9ea87ed1 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:24.454753 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d605506a-f8ba-4b6f-8d83-edbd9ea87ed1 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:24.455070 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d605506a-f8ba-4b6f-8d83-edbd9ea87ed1 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:24.456436 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 6/11] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:31:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:31:24.462599 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761c194-de5f-4bf8-8f25-cff12703b162 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:24.462919 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761c194-de5f-4bf8-8f25-cff12703b162 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:24.463207 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761c194-de5f-4bf8-8f25-cff12703b162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:24.472510 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 6/12] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:31:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:31:24.538734 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG passlib.handlers.bcrypt [None req-4761c194-de5f-4bf8-8f25-cff12703b162 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=88064) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 22 02:31:24.539448 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG passlib.handlers.bcrypt [None req-4761c194-de5f-4bf8-8f25-cff12703b162 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88064) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 22 02:31:24.596392 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-4761c194-de5f-4bf8-8f25-cff12703b162 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:31:24.599318 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-4761c194-de5f-4bf8-8f25-cff12703b162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:31:24.736527 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 7/13] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:31:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 02:31:24.816521 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448644c7-3269-4aa6-afd3-377a0e5f3aa5 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:24.851407 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:24.990498 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448644c7-3269-4aa6-afd3-377a0e5f3aa5 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:25.010333 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4761c194-de5f-4bf8-8f25-cff12703b162 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:25.036591 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edcf4a0-9675-4af7-9b53-079fb541be7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:25.037261 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edcf4a0-9675-4af7-9b53-079fb541be7d None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:25.037650 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edcf4a0-9675-4af7-9b53-079fb541be7d None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:25.037939 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edcf4a0-9675-4af7-9b53-079fb541be7d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:25.055378 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edcf4a0-9675-4af7-9b53-079fb541be7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:31:25.072348 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-0edcf4a0-9675-4af7-9b53-079fb541be7d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:31:25.072348 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:31:25.072348 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:31:25.072348 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:31:25.072348 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:31:25.072348 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:31:25.072348 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:31:25.072348 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:31:25.072348 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:31:25.072348 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:31:25.072348 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:31:25.072348 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:31:25.072348 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:31:25.072348 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:31:25.072348 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:31:25.072348 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:31:25.073727 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:31:25.073727 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:31:25.073727 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:31:25.073727 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:31:25.073727 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:31:25.073727 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:31:25.073727 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:31:25.073727 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:31:25.073727 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:31:25.073727 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:31:25.073727 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:31:25.073727 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:31:25.073727 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:31:25.073727 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:31:25.073727 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:31:25.073727 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:31:25.075284 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:31:25.075284 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:31:25.075284 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:31:25.075284 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:31:25.075284 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:31:25.075284 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:31:25.075284 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:31:25.075284 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:31:25.075284 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:31:25.075284 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:31:25.075284 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:31:25.075284 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:31:25.075284 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 7/14] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Sep 22 02:31:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:31:25.080660 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edcf4a0-9675-4af7-9b53-079fb541be7d None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:25.138814 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4761c194-de5f-4bf8-8f25-cff12703b162 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:25.141195 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:25.183015 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f55d92e-810b-4830-8ae5-6c5088c8bdf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:25.183905 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f55d92e-810b-4830-8ae5-6c5088c8bdf2 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:25.184208 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f55d92e-810b-4830-8ae5-6c5088c8bdf2 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:25.184484 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f55d92e-810b-4830-8ae5-6c5088c8bdf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:25.189604 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:25.210521 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edcf4a0-9675-4af7-9b53-079fb541be7d None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:25.249235 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf854eb-4bbf-45eb-9ec8-029b0cd51ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:25.249875 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf854eb-4bbf-45eb-9ec8-029b0cd51ebf None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:25.250163 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf854eb-4bbf-45eb-9ec8-029b0cd51ebf None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:25.250449 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf854eb-4bbf-45eb-9ec8-029b0cd51ebf None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:25.251198 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf854eb-4bbf-45eb-9ec8-029b0cd51ebf None admin] RBAC: Adding query filter params (name=admin) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:31:25.251499 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf854eb-4bbf-45eb-9ec8-029b0cd51ebf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:25.252941 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf854eb-4bbf-45eb-9ec8-029b0cd51ebf None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:25.263806 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f55d92e-810b-4830-8ae5-6c5088c8bdf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:25.265394 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f55d92e-810b-4830-8ae5-6c5088c8bdf2 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:25.266280 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 8/15] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:31:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:31:25.272817 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf854eb-4bbf-45eb-9ec8-029b0cd51ebf None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:25.378481 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 8/16] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:31:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 22 02:31:25.384788 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f55d92e-810b-4830-8ae5-6c5088c8bdf2 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:25.385181 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf854eb-4bbf-45eb-9ec8-029b0cd51ebf None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:25.388619 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:25.424096 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43478628-a4d0-433b-ba68-ef49e0080fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:25.424654 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43478628-a4d0-433b-ba68-ef49e0080fea None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:25.424940 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43478628-a4d0-433b-ba68-ef49e0080fea None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:25.425221 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43478628-a4d0-433b-ba68-ef49e0080fea None admin] PATH_INFO: `/v3/users/778bd65251774785bdd2460a6778044c` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:25.426501 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43478628-a4d0-433b-ba68-ef49e0080fea None admin] RBAC: Authorizing `identity:get_user(user_id=778bd65251774785bdd2460a6778044c)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:25.428034 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43478628-a4d0-433b-ba68-ef49e0080fea None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:25.429661 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 9/17] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:31:25 2022] GET /identity/v3/users/778bd65251774785bdd2460a6778044c => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:25.435197 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43478628-a4d0-433b-ba68-ef49e0080fea None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:25.452819 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:25.493459 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f55d92e-810b-4830-8ae5-6c5088c8bdf2 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:25.528670 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6df6ac-0b13-4797-a09c-958a07ceb621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:25.530155 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6df6ac-0b13-4797-a09c-958a07ceb621 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:25.530451 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6df6ac-0b13-4797-a09c-958a07ceb621 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:25.530732 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6df6ac-0b13-4797-a09c-958a07ceb621 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:25.540597 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43478628-a4d0-433b-ba68-ef49e0080fea None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:25.552060 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6df6ac-0b13-4797-a09c-958a07ceb621 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:25.553556 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6df6ac-0b13-4797-a09c-958a07ceb621 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:25.573144 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-cf6df6ac-0b13-4797-a09c-958a07ceb621 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:31:25.573144 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:31:25.573144 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:31:25.573144 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:31:25.573144 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:31:25.573144 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:31:25.573144 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:31:25.573144 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:31:25.573144 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:31:25.573144 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:31:25.573144 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:31:25.573144 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:31:25.573144 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:31:25.573144 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:31:25.573144 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:31:25.573144 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:31:25.574159 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:31:25.574159 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:31:25.574159 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:31:25.574159 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:31:25.574159 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:31:25.574159 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:31:25.574159 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:31:25.574159 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:31:25.574159 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:31:25.574159 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:31:25.574159 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:31:25.574159 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:31:25.574159 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:31:25.574159 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:31:25.574159 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:31:25.574159 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:31:25.574159 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:31:25.575030 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:31:25.575030 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:31:25.575030 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:31:25.575030 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 9/18] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:31:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:31:25.580737 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6df6ac-0b13-4797-a09c-958a07ceb621 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:25.581303 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e171ac-7a94-4bcc-8aa0-5550bbc5a37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:25.581949 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e171ac-7a94-4bcc-8aa0-5550bbc5a37b None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:25.582238 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e171ac-7a94-4bcc-8aa0-5550bbc5a37b None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:25.582513 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e171ac-7a94-4bcc-8aa0-5550bbc5a37b None admin] PATH_INFO: `/v3/domains/default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:25.583681 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e171ac-7a94-4bcc-8aa0-5550bbc5a37b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:25.585136 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e171ac-7a94-4bcc-8aa0-5550bbc5a37b None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:25.586874 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 10/19] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:31:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:31:25.593096 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e171ac-7a94-4bcc-8aa0-5550bbc5a37b None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:25.639429 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:25.677333 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:25.687470 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6df6ac-0b13-4797-a09c-958a07ceb621 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:25.697022 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e171ac-7a94-4bcc-8aa0-5550bbc5a37b None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:25.723934 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee7fe5-2c2a-4b30-8782-e42febc3498a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:25.724559 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee7fe5-2c2a-4b30-8782-e42febc3498a None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:25.724837 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee7fe5-2c2a-4b30-8782-e42febc3498a None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:25.725109 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee7fe5-2c2a-4b30-8782-e42febc3498a None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:25.725753 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee7fe5-2c2a-4b30-8782-e42febc3498a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:31:25.726056 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee7fe5-2c2a-4b30-8782-e42febc3498a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:25.727490 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee7fe5-2c2a-4b30-8782-e42febc3498a None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:25.731511 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a58a4-6f98-4547-ac99-58f79947b65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:25.732138 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a58a4-6f98-4547-ac99-58f79947b65f None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:25.732423 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a58a4-6f98-4547-ac99-58f79947b65f None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:25.732740 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a58a4-6f98-4547-ac99-58f79947b65f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:25.733470 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a58a4-6f98-4547-ac99-58f79947b65f None admin] RBAC: Adding query filter params (role.id=31a687548f55493b8ea666fe7e350303, scope.domain.id=default, user.id=778bd65251774785bdd2460a6778044c) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:31:25.733783 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a58a4-6f98-4547-ac99-58f79947b65f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:25.735259 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a58a4-6f98-4547-ac99-58f79947b65f None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:25.744539 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 11/20] 10.222.0.20 () {56 vars in 1322 bytes} [Thu Sep 22 02:31:25 2022] GET /identity/v3/role_assignments?user.id=778bd65251774785bdd2460a6778044c&scope.domain.id=default&role.id=31a687548f55493b8ea666fe7e350303 => generated 233 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:31:25.745179 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 10/21] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:31:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:31:28.339251 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4599ca-b965-4a81-a847-fcb0ddb942b0 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:28.339781 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4599ca-b965-4a81-a847-fcb0ddb942b0 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:28.339854 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4599ca-b965-4a81-a847-fcb0ddb942b0 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:28.341314 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 12/22] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:31:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:31:28.347892 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b0b9f-38d3-48da-bb5d-746be376cbde None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:28.348415 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b0b9f-38d3-48da-bb5d-746be376cbde None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:28.348511 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b0b9f-38d3-48da-bb5d-746be376cbde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:28.380449 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-399b0b9f-38d3-48da-bb5d-746be376cbde None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:31:28.382239 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-399b0b9f-38d3-48da-bb5d-746be376cbde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:31:28.500980 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 11/23] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:31:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:31:28.599851 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded3629-1f1c-4dbd-83ac-4b08e0a7f190 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:28.600056 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded3629-1f1c-4dbd-83ac-4b08e0a7f190 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:28.600491 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded3629-1f1c-4dbd-83ac-4b08e0a7f190 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:28.601999 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 13/24] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:31:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:31:28.608495 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15b0ab-a4e1-45f8-8273-4d2bc94e9b0e None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:28.609143 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15b0ab-a4e1-45f8-8273-4d2bc94e9b0e None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:28.609622 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15b0ab-a4e1-45f8-8273-4d2bc94e9b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:28.643924 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-aa15b0ab-a4e1-45f8-8273-4d2bc94e9b0e None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:31:28.645142 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-aa15b0ab-a4e1-45f8-8273-4d2bc94e9b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:31:28.699030 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:28.731497 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 12/25] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:31:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 02:31:28.802325 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd422f-e323-44e8-bff6-14bef14b964f None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:28.803017 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd422f-e323-44e8-bff6-14bef14b964f None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:28.803496 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd422f-e323-44e8-bff6-14bef14b964f None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:28.805039 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 14/26] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:31:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:31:28.811159 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f9b5e-9135-4a03-9000-912e495b8b6f None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:28.811457 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f9b5e-9135-4a03-9000-912e495b8b6f None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:28.811734 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f9b5e-9135-4a03-9000-912e495b8b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:28.839904 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-6a2f9b5e-9135-4a03-9000-912e495b8b6f None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:31:28.841135 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-6a2f9b5e-9135-4a03-9000-912e495b8b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:31:28.924930 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 13/27] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:31:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:31:28.959328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd422f-e323-44e8-bff6-14bef14b964f None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:29.116524 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd422f-e323-44e8-bff6-14bef14b964f None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:29.159823 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73b365-e531-4d85-86fe-350c7bcb2561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:29.160848 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73b365-e531-4d85-86fe-350c7bcb2561 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:29.161328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73b365-e531-4d85-86fe-350c7bcb2561 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:29.161813 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73b365-e531-4d85-86fe-350c7bcb2561 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:29.184263 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f73b365-e531-4d85-86fe-350c7bcb2561 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:31:29.190549 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-6f73b365-e531-4d85-86fe-350c7bcb2561 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:31:29.190549 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:31:29.190549 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:31:29.190549 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:31:29.190549 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:31:29.190549 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:31:29.190549 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:31:29.190549 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:31:29.190549 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:31:29.190549 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:31:29.190549 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:31:29.190549 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:31:29.190549 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:31:29.190549 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:31:29.190549 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:31:29.190549 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:31:29.191534 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:31:29.191534 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:31:29.191534 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:31:29.191534 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:31:29.191534 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:31:29.191534 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:31:29.191534 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:31:29.191534 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:31:29.191534 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:31:29.191534 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:31:29.191534 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:31:29.191534 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:31:29.191534 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:31:29.191534 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:31:29.191534 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:31:29.191534 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:31:29.192320 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:31:29.192320 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:31:29.192320 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:31:29.192320 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:31:29.192320 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:31:29.192320 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:31:29.192320 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:31:29.192320 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:31:29.192320 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:31:29.192320 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:31:29.192320 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:31:29.192320 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:31:29.194842 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 15/28] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Sep 22 02:31:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:31:29.201530 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f9b5e-9135-4a03-9000-912e495b8b6f None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:29.232055 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73b365-e531-4d85-86fe-350c7bcb2561 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:29.238427 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:29.358161 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f9b5e-9135-4a03-9000-912e495b8b6f None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:29.367080 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:29.400403 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73b365-e531-4d85-86fe-350c7bcb2561 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:29.403409 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8cb9f-c316-428c-a5d4-4e0f49b67ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:29.404040 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8cb9f-c316-428c-a5d4-4e0f49b67ac3 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:29.404885 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8cb9f-c316-428c-a5d4-4e0f49b67ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:29.405159 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8cb9f-c316-428c-a5d4-4e0f49b67ac3 None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:29.405830 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef8cb9f-c316-428c-a5d4-4e0f49b67ac3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:31:29.406137 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef8cb9f-c316-428c-a5d4-4e0f49b67ac3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:29.407552 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef8cb9f-c316-428c-a5d4-4e0f49b67ac3 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:29.423004 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 14/29] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:31:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:29.424932 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8cb9f-c316-428c-a5d4-4e0f49b67ac3 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:29.447830 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2f1c4-bc65-4099-a606-7b1bac70eb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:29.448964 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2f1c4-bc65-4099-a606-7b1bac70eb1d None admin] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:29.449398 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2f1c4-bc65-4099-a606-7b1bac70eb1d None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:29.449824 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2f1c4-bc65-4099-a606-7b1bac70eb1d None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:29.450692 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2f1c4-bc65-4099-a606-7b1bac70eb1d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:29.453298 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2f1c4-bc65-4099-a606-7b1bac70eb1d None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:29.547433 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8cb9f-c316-428c-a5d4-4e0f49b67ac3 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:29.588407 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380587d-2dfc-42de-a22f-5e2de2644877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:29.589492 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380587d-2dfc-42de-a22f-5e2de2644877 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:29.589975 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380587d-2dfc-42de-a22f-5e2de2644877 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:29.590443 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380587d-2dfc-42de-a22f-5e2de2644877 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:29.602264 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0380587d-2dfc-42de-a22f-5e2de2644877 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:29.604059 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0380587d-2dfc-42de-a22f-5e2de2644877 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:29.616301 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 16/30] 10.222.0.20 () {60 vars in 1131 bytes} [Thu Sep 22 02:31:29 2022] POST /identity/v3/roles => generated 225 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 02:31:29.618027 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef0266-a6b5-4c03-a493-77f70de495fc None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:29.618444 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef0266-a6b5-4c03-a493-77f70de495fc None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:29.618886 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef0266-a6b5-4c03-a493-77f70de495fc None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:29.620710 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 17/31] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:31:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:31:29.621662 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef0266-a6b5-4c03-a493-77f70de495fc None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:29.628212 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-0380587d-2dfc-42de-a22f-5e2de2644877 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:31:29.628212 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:31:29.628212 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:31:29.628212 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:31:29.628212 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:31:29.628212 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:31:29.628212 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:31:29.628212 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:31:29.628212 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:31:29.628212 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:31:29.628212 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:31:29.628212 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:31:29.628212 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:31:29.628212 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:31:29.628212 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:31:29.628212 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:31:29.629184 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:31:29.629184 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:31:29.629184 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:31:29.629184 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:31:29.629184 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:31:29.629184 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:31:29.629184 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:31:29.629184 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:31:29.629184 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:31:29.629184 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:31:29.629184 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:31:29.629184 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:31:29.629184 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:31:29.629184 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:31:29.629184 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:31:29.629184 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:31:29.629184 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:31:29.630029 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:31:29.630029 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:31:29.630029 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:31:29.633570 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 15/32] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:31:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:31:29.645490 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7433e-268f-4b89-a5ce-1b7a58f8ba31 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:29.646139 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7433e-268f-4b89-a5ce-1b7a58f8ba31 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:29.646621 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7433e-268f-4b89-a5ce-1b7a58f8ba31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:29.669641 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-46b7433e-268f-4b89-a5ce-1b7a58f8ba31 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:31:29.671322 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-46b7433e-268f-4b89-a5ce-1b7a58f8ba31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:31:29.691401 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:29.790041 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 16/33] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:31:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:31:29.790880 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7433e-268f-4b89-a5ce-1b7a58f8ba31 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:29.804505 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:29.928682 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef0266-a6b5-4c03-a493-77f70de495fc None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:29.952919 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7433e-268f-4b89-a5ce-1b7a58f8ba31 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:29.974228 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686c3a7-686f-43f7-816c-249935965b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:29.975305 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686c3a7-686f-43f7-816c-249935965b9e None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:29.975661 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686c3a7-686f-43f7-816c-249935965b9e None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:29.976011 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686c3a7-686f-43f7-816c-249935965b9e None admin] PATH_INFO: `/v3/users/778bd65251774785bdd2460a6778044c` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:29.979372 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1686c3a7-686f-43f7-816c-249935965b9e None admin] RBAC: Authorizing `identity:get_user(user_id=778bd65251774785bdd2460a6778044c)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:29.981045 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1686c3a7-686f-43f7-816c-249935965b9e None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:29.984078 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 18/34] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:31:29 2022] GET /identity/v3/users/778bd65251774785bdd2460a6778044c => generated 247 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:29.985478 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462425c-c8e8-4ce7-95b8-00fa06f8d8a7 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:29.985766 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462425c-c8e8-4ce7-95b8-00fa06f8d8a7 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:29.986043 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462425c-c8e8-4ce7-95b8-00fa06f8d8a7 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:29.988357 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 19/35] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:31:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:31:29.989370 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462425c-c8e8-4ce7-95b8-00fa06f8d8a7 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:30.005624 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91930f1-55b9-4ad5-a0d8-04106e1d6937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:30.007366 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91930f1-55b9-4ad5-a0d8-04106e1d6937 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:30.007888 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91930f1-55b9-4ad5-a0d8-04106e1d6937 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:30.008368 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91930f1-55b9-4ad5-a0d8-04106e1d6937 None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:30.009232 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91930f1-55b9-4ad5-a0d8-04106e1d6937 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:31:30.010233 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91930f1-55b9-4ad5-a0d8-04106e1d6937 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:30.012589 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91930f1-55b9-4ad5-a0d8-04106e1d6937 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:30.036241 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 17/36] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:31:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:30.038597 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546b85c-d43b-473b-9418-3d844cf11afe None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:30.039382 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546b85c-d43b-473b-9418-3d844cf11afe None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:30.040365 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546b85c-d43b-473b-9418-3d844cf11afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:30.068312 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-4546b85c-d43b-473b-9418-3d844cf11afe None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:31:30.070507 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-4546b85c-d43b-473b-9418-3d844cf11afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:31:30.075296 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:30.139307 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462425c-c8e8-4ce7-95b8-00fa06f8d8a7 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:30.159030 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 18/37] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:31:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:31:30.160650 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4546b85c-d43b-473b-9418-3d844cf11afe None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:30.173308 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:30.179541 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91928eb0-5bba-47bd-974f-85c2337d2b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:30.180500 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91928eb0-5bba-47bd-974f-85c2337d2b52 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:30.180990 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91928eb0-5bba-47bd-974f-85c2337d2b52 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:30.181486 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91928eb0-5bba-47bd-974f-85c2337d2b52 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:30.182970 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91928eb0-5bba-47bd-974f-85c2337d2b52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:30.185022 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91928eb0-5bba-47bd-974f-85c2337d2b52 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:30.186924 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 20/38] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:31:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:30.187894 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91928eb0-5bba-47bd-974f-85c2337d2b52 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:30.289519 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4546b85c-d43b-473b-9418-3d844cf11afe None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:30.314209 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91928eb0-5bba-47bd-974f-85c2337d2b52 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:30.335495 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf33b9-28df-4555-a87a-fe9139e00413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:30.336155 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf33b9-28df-4555-a87a-fe9139e00413 None admin] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:30.336447 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf33b9-28df-4555-a87a-fe9139e00413 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:30.336720 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf33b9-28df-4555-a87a-fe9139e00413 None admin] PATH_INFO: `/v3/projects` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:30.337686 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daf33b9-28df-4555-a87a-fe9139e00413 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:30.339567 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daf33b9-28df-4555-a87a-fe9139e00413 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:30.354933 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1539d410-c68e-4eea-bdbf-54dc71d9c5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:30.355122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:30.355984 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1539d410-c68e-4eea-bdbf-54dc71d9c5a4 None admin] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:30.356476 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1539d410-c68e-4eea-bdbf-54dc71d9c5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:30.357039 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1539d410-c68e-4eea-bdbf-54dc71d9c5a4 None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:30.357953 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1539d410-c68e-4eea-bdbf-54dc71d9c5a4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:30.359595 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1539d410-c68e-4eea-bdbf-54dc71d9c5a4 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:30.379104 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 21/39] 10.222.0.20 () {60 vars in 1131 bytes} [Thu Sep 22 02:31:30 2022] POST /identity/v3/roles => generated 219 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:31:30.380368 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1539d410-c68e-4eea-bdbf-54dc71d9c5a4 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:30.467904 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:30.504677 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 19/40] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Sep 22 02:31:30 2022] POST /identity/v3/projects => generated 301 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:31:30.507099 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75278ae-c370-4623-b5b4-92a57d8dafbf None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:30.507477 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75278ae-c370-4623-b5b4-92a57d8dafbf None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:30.507835 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75278ae-c370-4623-b5b4-92a57d8dafbf None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:30.509597 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 20/41] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:31:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:31:30.510299 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75278ae-c370-4623-b5b4-92a57d8dafbf None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:30.533920 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1539d410-c68e-4eea-bdbf-54dc71d9c5a4 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:30.577096 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa931d-a110-4513-b821-8ec876be3aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:30.577809 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa931d-a110-4513-b821-8ec876be3aea None admin] REQUEST_METHOD: `PUT` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:30.578099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa931d-a110-4513-b821-8ec876be3aea None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:30.578379 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa931d-a110-4513-b821-8ec876be3aea None admin] PATH_INFO: `/v3/domains/default/users/778bd65251774785bdd2460a6778044c/roles/31a687548f55493b8ea666fe7e350303` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:30.581182 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffa931d-a110-4513-b821-8ec876be3aea None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=778bd65251774785bdd2460a6778044c, role_id=31a687548f55493b8ea666fe7e350303)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:30.582695 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffa931d-a110-4513-b821-8ec876be3aea None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:30.610503 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 22/42] 10.222.0.20 () {58 vars in 1361 bytes} [Thu Sep 22 02:31:30 2022] PUT /identity/v3/domains/default/users/778bd65251774785bdd2460a6778044c/roles/31a687548f55493b8ea666fe7e350303 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 02:31:30.612167 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f111d9d-6b89-4512-93e8-65c0f5e0d7ed None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:30.612470 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f111d9d-6b89-4512-93e8-65c0f5e0d7ed None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:30.612751 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f111d9d-6b89-4512-93e8-65c0f5e0d7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:30.636711 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-4f111d9d-6b89-4512-93e8-65c0f5e0d7ed None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:31:30.638426 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-4f111d9d-6b89-4512-93e8-65c0f5e0d7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:31:30.680055 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75278ae-c370-4623-b5b4-92a57d8dafbf None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:30.730620 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e3d90-b7dc-40be-8a96-11aaf06203bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:30.731313 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e3d90-b7dc-40be-8a96-11aaf06203bc None admin] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:30.731600 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e3d90-b7dc-40be-8a96-11aaf06203bc None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:30.731880 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e3d90-b7dc-40be-8a96-11aaf06203bc None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:30.735561 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e3d90-b7dc-40be-8a96-11aaf06203bc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:30.737274 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e3d90-b7dc-40be-8a96-11aaf06203bc None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:30.741567 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 23/43] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:31:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:31:30.758366 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 21/44] 10.222.0.20 () {60 vars in 1131 bytes} [Thu Sep 22 02:31:30 2022] POST /identity/v3/roles => generated 223 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:31:31.025828 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f111d9d-6b89-4512-93e8-65c0f5e0d7ed None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:31.074845 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:31.126572 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f111d9d-6b89-4512-93e8-65c0f5e0d7ed None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:31.162201 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d14db4a-dce9-4cf1-8a12-617f0805fd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:31.162865 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d14db4a-dce9-4cf1-8a12-617f0805fd83 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:31.163155 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d14db4a-dce9-4cf1-8a12-617f0805fd83 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:31.163430 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d14db4a-dce9-4cf1-8a12-617f0805fd83 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:31.164549 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d14db4a-dce9-4cf1-8a12-617f0805fd83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:31.165990 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d14db4a-dce9-4cf1-8a12-617f0805fd83 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:31.167588 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 24/45] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:31:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:31.174093 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e3d90-b7dc-40be-8a96-11aaf06203bc None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:31.227882 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:31.276280 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e3d90-b7dc-40be-8a96-11aaf06203bc None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:31.310342 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8b68d5-699f-4116-aa61-546f15304a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:31.311019 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8b68d5-699f-4116-aa61-546f15304a3f None admin] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:31.311310 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8b68d5-699f-4116-aa61-546f15304a3f None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:31.311590 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8b68d5-699f-4116-aa61-546f15304a3f None admin] PATH_INFO: `/v3/projects` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:31.312257 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8b68d5-699f-4116-aa61-546f15304a3f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:31.313718 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8b68d5-699f-4116-aa61-546f15304a3f None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:31.342968 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 22/46] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Sep 22 02:31:31 2022] POST /identity/v3/projects => generated 312 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:31:32.861962 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e7701-4f5d-4093-9046-4f0bdb981af8 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:32.862533 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e7701-4f5d-4093-9046-4f0bdb981af8 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:32.862972 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e7701-4f5d-4093-9046-4f0bdb981af8 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:32.865429 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 25/47] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:31:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:31:32.873926 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975df27-5e30-4b5c-bfe4-ac8a10529eec None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:32.874678 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975df27-5e30-4b5c-bfe4-ac8a10529eec None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:32.875058 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975df27-5e30-4b5c-bfe4-ac8a10529eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:32.894616 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-f975df27-5e30-4b5c-bfe4-ac8a10529eec None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:31:32.896339 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-f975df27-5e30-4b5c-bfe4-ac8a10529eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:31:32.975984 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 23/48] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:31:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 02:31:33.503306 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e7701-4f5d-4093-9046-4f0bdb981af8 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:33.576516 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04982323-c5c7-45fe-8f91-049bf60133f6 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:33.576943 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04982323-c5c7-45fe-8f91-049bf60133f6 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:33.577376 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04982323-c5c7-45fe-8f91-049bf60133f6 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:33.579609 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 24/49] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:31:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:31:33.588036 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709d692-0dd6-47ed-89c6-e9a336e40c29 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:33.588494 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709d692-0dd6-47ed-89c6-e9a336e40c29 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:33.588928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709d692-0dd6-47ed-89c6-e9a336e40c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:33.617310 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-a709d692-0dd6-47ed-89c6-e9a336e40c29 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:31:33.619223 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-a709d692-0dd6-47ed-89c6-e9a336e40c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:31:33.664731 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:33.718382 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 25/50] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:31:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 02:31:33.757811 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e7701-4f5d-4093-9046-4f0bdb981af8 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:33.782359 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:33.814602 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d47a959-ffd3-4a78-878b-32a458d65c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:33.815672 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47a959-ffd3-4a78-878b-32a458d65c79 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:33.816166 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47a959-ffd3-4a78-878b-32a458d65c79 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:33.816635 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47a959-ffd3-4a78-878b-32a458d65c79 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:33.834105 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d47a959-ffd3-4a78-878b-32a458d65c79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:31:33.843815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-1d47a959-ffd3-4a78-878b-32a458d65c79 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:31:33.843815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:31:33.843815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:31:33.843815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:31:33.843815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:31:33.843815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:31:33.843815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:31:33.843815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:31:33.843815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:31:33.843815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:31:33.843815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:31:33.843815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:31:33.843815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:31:33.843815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:31:33.843815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:31:33.843815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:31:33.845570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:31:33.845570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:31:33.845570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:31:33.845570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:31:33.845570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:31:33.845570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:31:33.845570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:31:33.845570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:31:33.845570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:31:33.845570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:31:33.845570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:31:33.845570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:31:33.845570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:31:33.845570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:31:33.845570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:31:33.845570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:31:33.847118 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:31:33.847118 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:31:33.847118 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:31:33.847118 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:31:33.847118 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:31:33.847118 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:31:33.847118 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:31:33.847118 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:31:33.847118 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:31:33.847118 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:31:33.847118 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:31:33.847118 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:31:33.847118 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 26/51] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Sep 22 02:31:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:31:33.852840 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a709d692-0dd6-47ed-89c6-e9a336e40c29 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:34.044032 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d47a959-ffd3-4a78-878b-32a458d65c79 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:34.080840 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a709d692-0dd6-47ed-89c6-e9a336e40c29 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:34.132519 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b97a6f-87a7-4a5a-9658-ec24bd767393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:34.133152 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b97a6f-87a7-4a5a-9658-ec24bd767393 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:34.133440 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b97a6f-87a7-4a5a-9658-ec24bd767393 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:34.133720 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b97a6f-87a7-4a5a-9658-ec24bd767393 None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:34.134438 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b97a6f-87a7-4a5a-9658-ec24bd767393 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:31:34.134750 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b97a6f-87a7-4a5a-9658-ec24bd767393 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:34.136291 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b97a6f-87a7-4a5a-9658-ec24bd767393 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:34.145574 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 26/52] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:31:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:34.151940 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b97a6f-87a7-4a5a-9658-ec24bd767393 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:34.156639 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d47a959-ffd3-4a78-878b-32a458d65c79 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:34.182217 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:34.204330 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b0d166-d307-409d-a344-6c0ebf35ce05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:34.204982 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b0d166-d307-409d-a344-6c0ebf35ce05 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:34.205268 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b0d166-d307-409d-a344-6c0ebf35ce05 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:34.205558 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b0d166-d307-409d-a344-6c0ebf35ce05 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:34.206744 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b0d166-d307-409d-a344-6c0ebf35ce05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:34.208268 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b0d166-d307-409d-a344-6c0ebf35ce05 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:34.209884 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 27/53] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:31:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:31:34.215875 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b0d166-d307-409d-a344-6c0ebf35ce05 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:34.269903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:34.274717 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b97a6f-87a7-4a5a-9658-ec24bd767393 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:34.312109 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddacec6-41c6-48df-ae22-3ca0b5c68d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:34.312676 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddacec6-41c6-48df-ae22-3ca0b5c68d56 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:34.312966 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddacec6-41c6-48df-ae22-3ca0b5c68d56 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:34.313246 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddacec6-41c6-48df-ae22-3ca0b5c68d56 None admin] PATH_INFO: `/v3/users/778bd65251774785bdd2460a6778044c` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:34.314662 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddacec6-41c6-48df-ae22-3ca0b5c68d56 None admin] RBAC: Authorizing `identity:get_user(user_id=778bd65251774785bdd2460a6778044c)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:34.316160 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddacec6-41c6-48df-ae22-3ca0b5c68d56 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:34.317780 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 27/54] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:31:34 2022] GET /identity/v3/users/778bd65251774785bdd2460a6778044c => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:31:34.323435 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddacec6-41c6-48df-ae22-3ca0b5c68d56 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:34.357603 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b0d166-d307-409d-a344-6c0ebf35ce05 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:34.409507 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d514e-0c90-42a8-bb8c-ade76a8a5281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:34.410493 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d514e-0c90-42a8-bb8c-ade76a8a5281 None admin] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:34.411006 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d514e-0c90-42a8-bb8c-ade76a8a5281 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:34.411492 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d514e-0c90-42a8-bb8c-ade76a8a5281 None admin] PATH_INFO: `/v3/projects` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:34.412626 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4d514e-0c90-42a8-bb8c-ade76a8a5281 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:34.414952 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4d514e-0c90-42a8-bb8c-ade76a8a5281 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:34.454799 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:34.457897 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 28/55] 10.222.0.20 () {60 vars in 1140 bytes} [Thu Sep 22 02:31:34 2022] POST /identity/v3/projects => generated 298 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 02:31:34.464672 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddacec6-41c6-48df-ae22-3ca0b5c68d56 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:34.516012 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b8fd95-54db-4a43-b5f7-c3e2c61f0109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:34.516975 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8fd95-54db-4a43-b5f7-c3e2c61f0109 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:34.517427 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8fd95-54db-4a43-b5f7-c3e2c61f0109 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:34.517877 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8fd95-54db-4a43-b5f7-c3e2c61f0109 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:34.519850 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b8fd95-54db-4a43-b5f7-c3e2c61f0109 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:34.522441 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b8fd95-54db-4a43-b5f7-c3e2c61f0109 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:34.525287 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 28/56] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:31:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:31:34.532212 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d514e-0c90-42a8-bb8c-ade76a8a5281 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:34.657711 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:34.662449 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d514e-0c90-42a8-bb8c-ade76a8a5281 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:34.712239 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42eab9-721f-4ac4-b56a-f3d47b307fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:34.712930 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42eab9-721f-4ac4-b56a-f3d47b307fff None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:34.713383 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42eab9-721f-4ac4-b56a-f3d47b307fff None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:34.713836 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42eab9-721f-4ac4-b56a-f3d47b307fff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:34.714753 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42eab9-721f-4ac4-b56a-f3d47b307fff None admin] RBAC: Adding query filter params (role.id=31a687548f55493b8ea666fe7e350303, scope.domain.id=default, user.id=778bd65251774785bdd2460a6778044c) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:31:34.715216 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42eab9-721f-4ac4-b56a-f3d47b307fff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:34.717363 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42eab9-721f-4ac4-b56a-f3d47b307fff None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:34.725342 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 29/57] 10.222.0.20 () {56 vars in 1322 bytes} [Thu Sep 22 02:31:34 2022] GET /identity/v3/role_assignments?user.id=778bd65251774785bdd2460a6778044c&scope.domain.id=default&role.id=31a687548f55493b8ea666fe7e350303 => generated 530 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:36.743043 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705bebdd-61ce-4e9d-b1af-edc863d52ed4 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:36.743747 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705bebdd-61ce-4e9d-b1af-edc863d52ed4 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:36.743747 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705bebdd-61ce-4e9d-b1af-edc863d52ed4 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:36.744965 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 29/58] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:31:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:31:36.750979 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01b949-2f68-48af-b998-095beeade8a1 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:36.751487 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01b949-2f68-48af-b998-095beeade8a1 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:36.751571 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01b949-2f68-48af-b998-095beeade8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:36.771356 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-1d01b949-2f68-48af-b998-095beeade8a1 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:31:36.772606 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-1d01b949-2f68-48af-b998-095beeade8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:31:36.850705 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 30/59] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:31:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:31:37.131238 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705bebdd-61ce-4e9d-b1af-edc863d52ed4 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:37.252368 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705bebdd-61ce-4e9d-b1af-edc863d52ed4 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:37.288467 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f2652c-3f6c-46ed-a311-de5cb8615d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:37.289046 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f2652c-3f6c-46ed-a311-de5cb8615d22 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:37.289338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f2652c-3f6c-46ed-a311-de5cb8615d22 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:37.289620 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f2652c-3f6c-46ed-a311-de5cb8615d22 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:37.290970 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f2652c-3f6c-46ed-a311-de5cb8615d22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:37.292487 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f2652c-3f6c-46ed-a311-de5cb8615d22 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:37.294166 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 30/60] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:31:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:31:37.300764 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01b949-2f68-48af-b998-095beeade8a1 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:37.410021 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:37.420000 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01b949-2f68-48af-b998-095beeade8a1 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:37.458094 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705cae9f-ac7b-4c4e-8c94-185bae2046f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:37.458886 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705cae9f-ac7b-4c4e-8c94-185bae2046f5 None admin] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:37.459265 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705cae9f-ac7b-4c4e-8c94-185bae2046f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:37.459631 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705cae9f-ac7b-4c4e-8c94-185bae2046f5 None admin] PATH_INFO: `/v3/users` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:37.460395 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705cae9f-ac7b-4c4e-8c94-185bae2046f5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:37.461985 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705cae9f-ac7b-4c4e-8c94-185bae2046f5 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:37.491609 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 31/61] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Sep 22 02:31:37 2022] POST /identity/v3/users => generated 275 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 02:31:41.661950 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b913191-0c3c-4ef6-998f-90c7b3633791 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:41.663334 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b913191-0c3c-4ef6-998f-90c7b3633791 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:41.664111 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b913191-0c3c-4ef6-998f-90c7b3633791 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:41.666988 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 31/62] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:31:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:31:41.674779 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1adbf-2f1f-490d-ae0f-639e6876862a None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:41.676243 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1adbf-2f1f-490d-ae0f-639e6876862a None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:41.677055 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1adbf-2f1f-490d-ae0f-639e6876862a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:41.715271 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-dcf1adbf-2f1f-490d-ae0f-639e6876862a None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:31:41.718388 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-dcf1adbf-2f1f-490d-ae0f-639e6876862a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:31:41.913210 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 32/63] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:31:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:31:41.943876 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb383aa-68d9-4685-9502-b1ac72ab8c68 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:41.945089 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb383aa-68d9-4685-9502-b1ac72ab8c68 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:41.945972 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb383aa-68d9-4685-9502-b1ac72ab8c68 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:41.953924 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 32/64] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:31:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:31:41.956472 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696fd41-40e8-40a8-9111-cf9134d97282 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:41.956987 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696fd41-40e8-40a8-9111-cf9134d97282 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:41.957495 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696fd41-40e8-40a8-9111-cf9134d97282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:42.004671 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-0696fd41-40e8-40a8-9111-cf9134d97282 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:31:42.006764 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-0696fd41-40e8-40a8-9111-cf9134d97282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:31:42.204606 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 33/65] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:31:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 02:31:42.556052 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb383aa-68d9-4685-9502-b1ac72ab8c68 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:42.630003 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:42.649377 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f768935-b7a9-4653-b8c3-73ea4cc7ba03 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:42.649866 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f768935-b7a9-4653-b8c3-73ea4cc7ba03 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:42.650378 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f768935-b7a9-4653-b8c3-73ea4cc7ba03 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:42.653745 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 34/66] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:31:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:31:42.665180 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481957bf-94c9-4c0e-ac61-970a2bb76411 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:42.665676 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481957bf-94c9-4c0e-ac61-970a2bb76411 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:42.666177 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481957bf-94c9-4c0e-ac61-970a2bb76411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:42.718193 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-481957bf-94c9-4c0e-ac61-970a2bb76411 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:31:42.725954 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-481957bf-94c9-4c0e-ac61-970a2bb76411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:31:42.731208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:42.760011 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb383aa-68d9-4685-9502-b1ac72ab8c68 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:42.809670 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84893de4-58e3-493a-9de6-1efc9beddf9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:42.811784 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84893de4-58e3-493a-9de6-1efc9beddf9e None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:42.812918 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84893de4-58e3-493a-9de6-1efc9beddf9e None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:42.813744 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84893de4-58e3-493a-9de6-1efc9beddf9e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:42.842480 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84893de4-58e3-493a-9de6-1efc9beddf9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:31:42.844347 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 35/67] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:31:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:31:42.845488 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481957bf-94c9-4c0e-ac61-970a2bb76411 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:42.858360 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-84893de4-58e3-493a-9de6-1efc9beddf9e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:31:42.858360 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:31:42.858360 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:31:42.858360 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:31:42.858360 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:31:42.858360 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:31:42.858360 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:31:42.858360 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:31:42.858360 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:31:42.858360 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:31:42.858360 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:31:42.858360 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:31:42.858360 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:31:42.858360 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:31:42.858360 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:31:42.858360 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:31:42.860331 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:31:42.860331 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:31:42.860331 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:31:42.860331 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:31:42.860331 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:31:42.860331 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:31:42.860331 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:31:42.860331 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:31:42.860331 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:31:42.860331 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:31:42.860331 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:31:42.860331 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:31:42.860331 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:31:42.860331 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:31:42.860331 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:31:42.860331 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:31:42.862079 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:31:42.862079 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:31:42.862079 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:31:42.862079 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:31:42.862079 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:31:42.862079 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:31:42.862079 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:31:42.862079 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:31:42.862079 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:31:42.862079 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:31:42.862079 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:31:42.862079 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:31:42.866689 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 33/68] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Sep 22 02:31:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:31:42.878172 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84893de4-58e3-493a-9de6-1efc9beddf9e None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:43.051450 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:43.066961 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84893de4-58e3-493a-9de6-1efc9beddf9e None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:43.100992 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511239b1-d267-4422-a4d1-c54c94ddaa03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:43.101871 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511239b1-d267-4422-a4d1-c54c94ddaa03 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:43.102324 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511239b1-d267-4422-a4d1-c54c94ddaa03 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:43.102747 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511239b1-d267-4422-a4d1-c54c94ddaa03 None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:43.103605 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511239b1-d267-4422-a4d1-c54c94ddaa03 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:31:43.104064 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511239b1-d267-4422-a4d1-c54c94ddaa03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:43.105693 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511239b1-d267-4422-a4d1-c54c94ddaa03 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:43.105895 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481957bf-94c9-4c0e-ac61-970a2bb76411 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:43.115907 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 34/69] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:31:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:31:43.124048 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511239b1-d267-4422-a4d1-c54c94ddaa03 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:43.183377 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed58447-0e23-4a20-a910-bec9f40d6b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:43.184337 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed58447-0e23-4a20-a910-bec9f40d6b2a None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:43.185332 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed58447-0e23-4a20-a910-bec9f40d6b2a None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:43.185779 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed58447-0e23-4a20-a910-bec9f40d6b2a None admin] PATH_INFO: `/v3/roles/member` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:43.199772 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed58447-0e23-4a20-a910-bec9f40d6b2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:31:43.206322 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-4ed58447-0e23-4a20-a910-bec9f40d6b2a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 02:31:43.206322 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:31:43.206322 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:31:43.206322 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:31:43.206322 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:31:43.206322 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:31:43.206322 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:31:43.206322 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:31:43.206322 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:31:43.206322 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:31:43.206322 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:31:43.206322 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:31:43.206322 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:31:43.206322 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:31:43.206322 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:31:43.206322 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:31:43.207418 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:31:43.207418 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:31:43.207418 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:31:43.207418 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:31:43.207418 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:31:43.207418 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:31:43.207418 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:31:43.207418 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:31:43.207418 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:31:43.207418 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:31:43.207418 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:31:43.207418 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:31:43.207418 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:31:43.207418 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:31:43.207418 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:31:43.207418 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:31:43.208327 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:31:43.208327 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:31:43.208327 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:31:43.208327 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:31:43.208327 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:31:43.208327 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:31:43.208327 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:31:43.208327 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:31:43.208327 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:31:43.208327 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:31:43.208327 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 02:31:43.208327 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:31:43.210657 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 36/70] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:31:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:31:43.216028 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:43.220312 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9180d83-24d7-458e-ad89-6fb13d171f7e None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:43.220884 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9180d83-24d7-458e-ad89-6fb13d171f7e None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:43.221367 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9180d83-24d7-458e-ad89-6fb13d171f7e None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:43.224044 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 37/71] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:31:43 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:31:43.225574 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9180d83-24d7-458e-ad89-6fb13d171f7e None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:43.304311 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511239b1-d267-4422-a4d1-c54c94ddaa03 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:43.357069 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98fb0c7-afc2-4fee-93d7-62cf84e3f21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:43.358089 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98fb0c7-afc2-4fee-93d7-62cf84e3f21d None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:43.358599 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98fb0c7-afc2-4fee-93d7-62cf84e3f21d None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:43.359153 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98fb0c7-afc2-4fee-93d7-62cf84e3f21d None admin] PATH_INFO: `/v3/users/778bd65251774785bdd2460a6778044c` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:43.361216 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98fb0c7-afc2-4fee-93d7-62cf84e3f21d None admin] RBAC: Authorizing `identity:get_user(user_id=778bd65251774785bdd2460a6778044c)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:43.363992 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98fb0c7-afc2-4fee-93d7-62cf84e3f21d None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:43.367139 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 35/72] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:31:43 2022] GET /identity/v3/users/778bd65251774785bdd2460a6778044c => generated 247 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:43.369357 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1442be-830b-46c0-9c8c-0616b17e5ae9 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:43.369867 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1442be-830b-46c0-9c8c-0616b17e5ae9 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:43.370359 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1442be-830b-46c0-9c8c-0616b17e5ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:43.372794 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9180d83-24d7-458e-ad89-6fb13d171f7e None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:43.401856 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-6b1442be-830b-46c0-9c8c-0616b17e5ae9 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:31:43.404044 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-6b1442be-830b-46c0-9c8c-0616b17e5ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:31:43.420859 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c35c56-5f45-42b8-9d5f-39d7d40de704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:43.421929 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c35c56-5f45-42b8-9d5f-39d7d40de704 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:43.422533 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c35c56-5f45-42b8-9d5f-39d7d40de704 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:43.422965 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c35c56-5f45-42b8-9d5f-39d7d40de704 None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:43.423317 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:43.424392 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c35c56-5f45-42b8-9d5f-39d7d40de704 None admin] RBAC: Adding query filter params (name=member) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:31:43.424896 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c35c56-5f45-42b8-9d5f-39d7d40de704 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:43.428171 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c35c56-5f45-42b8-9d5f-39d7d40de704 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:43.442746 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 38/73] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Sep 22 02:31:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:43.444081 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00240aa-7a9d-40d0-b4a5-7a5e01158fb6 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:43.444389 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00240aa-7a9d-40d0-b4a5-7a5e01158fb6 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:43.444676 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00240aa-7a9d-40d0-b4a5-7a5e01158fb6 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:43.446244 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 39/74] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:31:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:31:43.446694 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00240aa-7a9d-40d0-b4a5-7a5e01158fb6 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:43.502908 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 36/75] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:31:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:31:43.504886 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1442be-830b-46c0-9c8c-0616b17e5ae9 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:43.523668 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:43.581451 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00240aa-7a9d-40d0-b4a5-7a5e01158fb6 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:43.625934 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1842eb-1366-4f10-b21d-958e0aea092d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:43.627328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1842eb-1366-4f10-b21d-958e0aea092d None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:43.627854 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1842eb-1366-4f10-b21d-958e0aea092d None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:43.628463 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1842eb-1366-4f10-b21d-958e0aea092d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:43.692215 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1442be-830b-46c0-9c8c-0616b17e5ae9 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:43.736851 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45061d8-555b-469d-809b-ec7ffddb8e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:43.737506 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45061d8-555b-469d-809b-ec7ffddb8e7a None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:43.737941 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45061d8-555b-469d-809b-ec7ffddb8e7a None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:43.738110 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45061d8-555b-469d-809b-ec7ffddb8e7a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:43.750237 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45061d8-555b-469d-809b-ec7ffddb8e7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:31:43.752289 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1842eb-1366-4f10-b21d-958e0aea092d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:43.755001 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1842eb-1366-4f10-b21d-958e0aea092d None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:43.755396 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-d45061d8-555b-469d-809b-ec7ffddb8e7a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 02:31:43.755396 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:31:43.755396 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:31:43.755396 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:31:43.755396 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:31:43.755396 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:31:43.755396 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:31:43.755396 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:31:43.755396 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:31:43.755396 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:31:43.755396 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:31:43.755396 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:31:43.755396 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:31:43.755396 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:31:43.755396 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:31:43.755396 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:31:43.759185 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:31:43.759185 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:31:43.759185 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:31:43.759185 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:31:43.759185 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:31:43.759185 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:31:43.759185 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:31:43.759185 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:31:43.759185 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:31:43.759185 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:31:43.759185 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:31:43.759185 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:31:43.759185 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:31:43.759185 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:31:43.759185 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:31:43.759185 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:31:43.761387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:31:43.761387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:31:43.761387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:31:43.761387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:31:43.761387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:31:43.761387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:31:43.761387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:31:43.761387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:31:43.761387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:31:43.761387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:31:43.761387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 02:31:43.761387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:31:43.761387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 37/76] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Sep 22 02:31:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:31:43.761387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11211d9-987d-4c68-95ed-7337175b8744 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:43.763255 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11211d9-987d-4c68-95ed-7337175b8744 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:43.763255 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11211d9-987d-4c68-95ed-7337175b8744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:43.780597 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-d11211d9-987d-4c68-95ed-7337175b8744 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:31:43.781864 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-d11211d9-987d-4c68-95ed-7337175b8744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:31:43.785687 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:43.874854 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:43.889751 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 38/77] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:31:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:31:43.889751 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11211d9-987d-4c68-95ed-7337175b8744 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:43.963675 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 40/78] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:31:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 02:31:43.963675 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1842eb-1366-4f10-b21d-958e0aea092d None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:44.064213 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11211d9-987d-4c68-95ed-7337175b8744 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:44.119187 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b5ea31-d843-47fb-bc2e-79f4ea74ee18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:44.120671 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5ea31-d843-47fb-bc2e-79f4ea74ee18 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:44.121185 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5ea31-d843-47fb-bc2e-79f4ea74ee18 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:44.121694 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5ea31-d843-47fb-bc2e-79f4ea74ee18 None admin] PATH_INFO: `/v3/users/c651e80ba10343f4ab505b7e2c378648` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:44.123534 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1842eb-1366-4f10-b21d-958e0aea092d None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:44.139393 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:44.151800 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b5ea31-d843-47fb-bc2e-79f4ea74ee18 None admin] RBAC: Authorizing `identity:get_user(user_id=c651e80ba10343f4ab505b7e2c378648)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:44.154733 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b5ea31-d843-47fb-bc2e-79f4ea74ee18 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:44.158884 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 39/79] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:31:43 2022] GET /identity/v3/users/c651e80ba10343f4ab505b7e2c378648 => generated 275 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:44.160145 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b5ea31-d843-47fb-bc2e-79f4ea74ee18 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:44.175167 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90178f-aa22-4fb9-8dc2-d0aa0ef8fa4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:44.175793 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90178f-aa22-4fb9-8dc2-d0aa0ef8fa4a None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:44.176077 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90178f-aa22-4fb9-8dc2-d0aa0ef8fa4a None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:44.176357 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90178f-aa22-4fb9-8dc2-d0aa0ef8fa4a None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:44.177073 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b90178f-aa22-4fb9-8dc2-d0aa0ef8fa4a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:31:44.177374 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b90178f-aa22-4fb9-8dc2-d0aa0ef8fa4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:44.178786 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b90178f-aa22-4fb9-8dc2-d0aa0ef8fa4a None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:44.188714 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 41/80] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:31:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:44.189671 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90178f-aa22-4fb9-8dc2-d0aa0ef8fa4a None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:44.221105 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:44.314535 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90178f-aa22-4fb9-8dc2-d0aa0ef8fa4a None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:44.327380 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b5ea31-d843-47fb-bc2e-79f4ea74ee18 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:44.354413 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec2314-b547-4408-a90d-883a43f095f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:44.355113 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec2314-b547-4408-a90d-883a43f095f3 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:44.355402 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec2314-b547-4408-a90d-883a43f095f3 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:44.355682 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec2314-b547-4408-a90d-883a43f095f3 None admin] PATH_INFO: `/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:44.357416 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fec2314-b547-4408-a90d-883a43f095f3 None admin] RBAC: Authorizing `identity:get_project(project_id=6a5cfaff6dc7495f9a0648c76c9f6eb7)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:44.358961 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fec2314-b547-4408-a90d-883a43f095f3 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:44.360903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 42/81] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:31:44 2022] GET /identity/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:44.361535 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec2314-b547-4408-a90d-883a43f095f3 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:44.366208 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2220b36-de39-4cd2-a7f6-32e6f2219bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:44.366890 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2220b36-de39-4cd2-a7f6-32e6f2219bcc None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:44.367178 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2220b36-de39-4cd2-a7f6-32e6f2219bcc None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:44.367457 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2220b36-de39-4cd2-a7f6-32e6f2219bcc None admin] PATH_INFO: `/v3/roles/member` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:44.379190 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2220b36-de39-4cd2-a7f6-32e6f2219bcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:31:44.387846 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-b2220b36-de39-4cd2-a7f6-32e6f2219bcc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 02:31:44.387846 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:31:44.387846 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:31:44.387846 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:31:44.387846 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:31:44.387846 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:31:44.387846 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:31:44.387846 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:31:44.387846 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:31:44.387846 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:31:44.387846 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:31:44.387846 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:31:44.387846 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:31:44.387846 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:31:44.387846 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:31:44.387846 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:31:44.388835 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:31:44.388835 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:31:44.388835 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:31:44.388835 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:31:44.388835 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:31:44.388835 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:31:44.388835 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:31:44.388835 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:31:44.388835 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:31:44.388835 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:31:44.388835 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:31:44.388835 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:31:44.388835 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:31:44.388835 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:31:44.388835 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:31:44.388835 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:31:44.391122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:31:44.391122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:31:44.391122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:31:44.391122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:31:44.391122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:31:44.391122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:31:44.391122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:31:44.391122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:31:44.391122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:31:44.391122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:31:44.391122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 02:31:44.391122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:31:44.391122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 40/82] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:31:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:31:44.391122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2220b36-de39-4cd2-a7f6-32e6f2219bcc None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:44.465030 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:44.507633 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:44.521204 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec2314-b547-4408-a90d-883a43f095f3 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:44.530352 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2220b36-de39-4cd2-a7f6-32e6f2219bcc None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:44.566264 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49a3eb-8f7b-4c26-a961-f9b82880dc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:44.567295 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49a3eb-8f7b-4c26-a961-f9b82880dc38 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:44.567804 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49a3eb-8f7b-4c26-a961-f9b82880dc38 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:44.568320 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49a3eb-8f7b-4c26-a961-f9b82880dc38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:44.581295 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1fb865-190b-471d-885c-82872ad0ac50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:44.582074 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1fb865-190b-471d-885c-82872ad0ac50 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:44.582496 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1fb865-190b-471d-885c-82872ad0ac50 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:44.582887 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1fb865-190b-471d-885c-82872ad0ac50 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:44.584372 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1fb865-190b-471d-885c-82872ad0ac50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:44.585981 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1fb865-190b-471d-885c-82872ad0ac50 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:44.587981 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 41/83] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:31:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:44.589399 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1fb865-190b-471d-885c-82872ad0ac50 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:44.663266 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c49a3eb-8f7b-4c26-a961-f9b82880dc38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:44.664802 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c49a3eb-8f7b-4c26-a961-f9b82880dc38 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:44.743325 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1fb865-190b-471d-885c-82872ad0ac50 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:44.769012 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:44.780562 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:44.784859 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 43/84] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:31:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 02:31:44.785758 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49a3eb-8f7b-4c26-a961-f9b82880dc38 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:44.800838 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2fe13-3596-42f7-b01d-4cd402c35489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:44.801830 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2fe13-3596-42f7-b01d-4cd402c35489 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:44.802274 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2fe13-3596-42f7-b01d-4cd402c35489 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:44.802708 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2fe13-3596-42f7-b01d-4cd402c35489 None admin] PATH_INFO: `/v3/users/c651e80ba10343f4ab505b7e2c378648` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:44.804722 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b2fe13-3596-42f7-b01d-4cd402c35489 None admin] RBAC: Authorizing `identity:get_user(user_id=c651e80ba10343f4ab505b7e2c378648)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:44.807742 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b2fe13-3596-42f7-b01d-4cd402c35489 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:44.811446 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 42/85] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:31:44 2022] GET /identity/v3/users/c651e80ba10343f4ab505b7e2c378648 => generated 275 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:44.813003 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2fe13-3596-42f7-b01d-4cd402c35489 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:44.924531 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49a3eb-8f7b-4c26-a961-f9b82880dc38 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:44.949073 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2fe13-3596-42f7-b01d-4cd402c35489 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:44.979671 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a56807-1a1d-4d7a-877d-d93aff86cbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:44.980378 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a56807-1a1d-4d7a-877d-d93aff86cbdc None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:44.980719 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a56807-1a1d-4d7a-877d-d93aff86cbdc None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:44.981060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a56807-1a1d-4d7a-877d-d93aff86cbdc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:44.982546 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a56807-1a1d-4d7a-877d-d93aff86cbdc None admin] RBAC: Adding query filter params (role.id=31a687548f55493b8ea666fe7e350303, scope.project.id=6a5cfaff6dc7495f9a0648c76c9f6eb7, user.id=778bd65251774785bdd2460a6778044c) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:31:44.983000 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a56807-1a1d-4d7a-877d-d93aff86cbdc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:44.984948 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a56807-1a1d-4d7a-877d-d93aff86cbdc None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:44.992084 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 44/86] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:31:44 2022] GET /identity/v3/role_assignments?user.id=778bd65251774785bdd2460a6778044c&scope.project.id=6a5cfaff6dc7495f9a0648c76c9f6eb7&role.id=31a687548f55493b8ea666fe7e350303 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:44.993625 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a56807-1a1d-4d7a-877d-d93aff86cbdc None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:45.000829 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb56c7-79d1-4ed7-8bb0-1aa1b7ac6c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:45.001661 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb56c7-79d1-4ed7-8bb0-1aa1b7ac6c48 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:45.002079 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb56c7-79d1-4ed7-8bb0-1aa1b7ac6c48 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:45.002486 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb56c7-79d1-4ed7-8bb0-1aa1b7ac6c48 None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:45.003589 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deb56c7-79d1-4ed7-8bb0-1aa1b7ac6c48 None admin] RBAC: Adding query filter params (name=member) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:31:45.004044 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deb56c7-79d1-4ed7-8bb0-1aa1b7ac6c48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:45.005734 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deb56c7-79d1-4ed7-8bb0-1aa1b7ac6c48 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:45.026290 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 43/87] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Sep 22 02:31:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:45.028165 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb56c7-79d1-4ed7-8bb0-1aa1b7ac6c48 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:45.102427 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:45.131896 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:45.189358 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a56807-1a1d-4d7a-877d-d93aff86cbdc None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:45.198753 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb56c7-79d1-4ed7-8bb0-1aa1b7ac6c48 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:45.247965 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0185415-55e3-406a-b774-bb30925a7646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:45.248713 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0185415-55e3-406a-b774-bb30925a7646 None admin] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:45.249157 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0185415-55e3-406a-b774-bb30925a7646 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:45.249578 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0185415-55e3-406a-b774-bb30925a7646 None admin] PATH_INFO: `/v3/users` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:45.250411 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0185415-55e3-406a-b774-bb30925a7646 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:45.250797 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb9c44-1624-475f-ac26-9d087126a83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:45.251615 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb9c44-1624-475f-ac26-9d087126a83d None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:45.251958 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb9c44-1624-475f-ac26-9d087126a83d None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:45.252294 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb9c44-1624-475f-ac26-9d087126a83d None admin] PATH_INFO: `/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:45.253737 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adb9c44-1624-475f-ac26-9d087126a83d None admin] RBAC: Authorizing `identity:get_project(project_id=6a5cfaff6dc7495f9a0648c76c9f6eb7)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:45.254162 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0185415-55e3-406a-b774-bb30925a7646 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:45.255647 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adb9c44-1624-475f-ac26-9d087126a83d None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:45.258478 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 44/88] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:31:45 2022] GET /identity/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7 => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:45.261077 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb9c44-1624-475f-ac26-9d087126a83d None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:45.287531 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 45/89] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Sep 22 02:31:44 2022] POST /identity/v3/users => generated 289 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:31:45.289773 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0185415-55e3-406a-b774-bb30925a7646 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:45.475890 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb9c44-1624-475f-ac26-9d087126a83d None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:45.523581 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0185415-55e3-406a-b774-bb30925a7646 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:45.531357 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:45.544571 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:45.552559 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b975b-a751-44f4-ba48-a0762271c48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:45.554108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b975b-a751-44f4-ba48-a0762271c48d None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:45.554425 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b975b-a751-44f4-ba48-a0762271c48d None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:45.554711 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b975b-a751-44f4-ba48-a0762271c48d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:45.594539 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c57aa-64ae-4cae-86c4-7bcf71cf25e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:45.595836 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c57aa-64ae-4cae-86c4-7bcf71cf25e5 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:45.596498 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c57aa-64ae-4cae-86c4-7bcf71cf25e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:45.597079 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c57aa-64ae-4cae-86c4-7bcf71cf25e5 None admin] PATH_INFO: `/v3/users/c651e80ba10343f4ab505b7e2c378648` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:45.599335 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c57aa-64ae-4cae-86c4-7bcf71cf25e5 None admin] RBAC: Authorizing `identity:get_user(user_id=c651e80ba10343f4ab505b7e2c378648)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:45.602400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c57aa-64ae-4cae-86c4-7bcf71cf25e5 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:45.605553 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 46/90] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:31:45 2022] GET /identity/v3/users/c651e80ba10343f4ab505b7e2c378648 => generated 275 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:45.607133 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c57aa-64ae-4cae-86c4-7bcf71cf25e5 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:45.654611 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011b975b-a751-44f4-ba48-a0762271c48d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:45.656211 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011b975b-a751-44f4-ba48-a0762271c48d None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:45.813368 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 45/91] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:31:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 02:31:45.813368 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b975b-a751-44f4-ba48-a0762271c48d None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:45.819686 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c57aa-64ae-4cae-86c4-7bcf71cf25e5 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:45.849343 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:45.897111 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22753bda-9060-4aa2-98ac-2fe2393e1801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:45.898190 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22753bda-9060-4aa2-98ac-2fe2393e1801 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:45.898708 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22753bda-9060-4aa2-98ac-2fe2393e1801 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:45.899274 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22753bda-9060-4aa2-98ac-2fe2393e1801 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:45.900966 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22753bda-9060-4aa2-98ac-2fe2393e1801 None admin] RBAC: Adding query filter params (role.id=83efac3b92eb44e783f4924c9d9ee842, scope.project.id=6a5cfaff6dc7495f9a0648c76c9f6eb7, user.id=c651e80ba10343f4ab505b7e2c378648) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:31:45.901341 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22753bda-9060-4aa2-98ac-2fe2393e1801 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:45.904950 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22753bda-9060-4aa2-98ac-2fe2393e1801 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:45.916524 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 47/92] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:31:45 2022] GET /identity/v3/role_assignments?user.id=c651e80ba10343f4ab505b7e2c378648&scope.project.id=6a5cfaff6dc7495f9a0648c76c9f6eb7&role.id=83efac3b92eb44e783f4924c9d9ee842 => generated 259 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:45.917923 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22753bda-9060-4aa2-98ac-2fe2393e1801 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:46.014487 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:46.017374 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b975b-a751-44f4-ba48-a0762271c48d None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:46.088766 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6050e-543d-4760-92f9-8b8df694424f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:46.089848 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6050e-543d-4760-92f9-8b8df694424f None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:46.090370 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6050e-543d-4760-92f9-8b8df694424f None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:46.091506 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6050e-543d-4760-92f9-8b8df694424f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:46.240798 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec6050e-543d-4760-92f9-8b8df694424f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:46.243620 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec6050e-543d-4760-92f9-8b8df694424f None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:46.248842 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22753bda-9060-4aa2-98ac-2fe2393e1801 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:46.307566 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:46.336165 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998901d1-a99d-463a-8f28-621baeb3d461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:46.337211 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998901d1-a99d-463a-8f28-621baeb3d461 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:46.337731 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998901d1-a99d-463a-8f28-621baeb3d461 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:46.338213 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998901d1-a99d-463a-8f28-621baeb3d461 None admin] PATH_INFO: `/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:46.339832 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998901d1-a99d-463a-8f28-621baeb3d461 None admin] RBAC: Authorizing `identity:get_project(project_id=6a5cfaff6dc7495f9a0648c76c9f6eb7)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:46.341913 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998901d1-a99d-463a-8f28-621baeb3d461 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:46.350975 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 48/93] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:31:45 2022] GET /identity/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7 => generated 298 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:46.353203 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998901d1-a99d-463a-8f28-621baeb3d461 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:46.440382 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 46/94] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:31:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 628 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 02:31:46.451061 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6050e-543d-4760-92f9-8b8df694424f None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:46.490736 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998901d1-a99d-463a-8f28-621baeb3d461 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:46.504623 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:46.548205 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177a54a-952d-4a12-a3c4-6859ba189ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:46.549282 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177a54a-952d-4a12-a3c4-6859ba189ff6 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:46.549778 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177a54a-952d-4a12-a3c4-6859ba189ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:46.550273 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177a54a-952d-4a12-a3c4-6859ba189ff6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:46.555610 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9177a54a-952d-4a12-a3c4-6859ba189ff6 None admin] RBAC: Adding query filter params (role.id=adec3004b56243fa923bd9f0082a2728, scope.project.id=6a5cfaff6dc7495f9a0648c76c9f6eb7, user.id=c651e80ba10343f4ab505b7e2c378648) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:31:46.556299 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9177a54a-952d-4a12-a3c4-6859ba189ff6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:46.558089 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9177a54a-952d-4a12-a3c4-6859ba189ff6 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:46.573477 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 49/95] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:31:46 2022] GET /identity/v3/role_assignments?user.id=c651e80ba10343f4ab505b7e2c378648&scope.project.id=6a5cfaff6dc7495f9a0648c76c9f6eb7&role.id=adec3004b56243fa923bd9f0082a2728 => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:46.683368 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6050e-543d-4760-92f9-8b8df694424f None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:46.733051 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:46.764136 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea0b63-b020-4068-b6e5-d6ef288d7566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:46.765269 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea0b63-b020-4068-b6e5-d6ef288d7566 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:46.765787 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea0b63-b020-4068-b6e5-d6ef288d7566 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:46.766288 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea0b63-b020-4068-b6e5-d6ef288d7566 None admin] PATH_INFO: `/v3/projects/5479a6e9aec54c4c9d3e119308cf2c88` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:46.768535 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ea0b63-b020-4068-b6e5-d6ef288d7566 None admin] RBAC: Authorizing `identity:get_project(project_id=5479a6e9aec54c4c9d3e119308cf2c88)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:46.771484 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ea0b63-b020-4068-b6e5-d6ef288d7566 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:46.774859 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 47/96] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:31:46 2022] GET /identity/v3/projects/5479a6e9aec54c4c9d3e119308cf2c88 => generated 312 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:46.784916 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177a54a-952d-4a12-a3c4-6859ba189ff6 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:47.042972 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177a54a-952d-4a12-a3c4-6859ba189ff6 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:47.118989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8db9df-aca7-454f-a48f-a7b0025cf71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:47.120054 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8db9df-aca7-454f-a48f-a7b0025cf71f None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:47.120530 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8db9df-aca7-454f-a48f-a7b0025cf71f None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:47.121076 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8db9df-aca7-454f-a48f-a7b0025cf71f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:47.123061 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8db9df-aca7-454f-a48f-a7b0025cf71f None admin] RBAC: Adding query filter params (role.id=83efac3b92eb44e783f4924c9d9ee842, scope.project.id=5479a6e9aec54c4c9d3e119308cf2c88, user.id=c651e80ba10343f4ab505b7e2c378648) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:31:47.123605 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8db9df-aca7-454f-a48f-a7b0025cf71f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:47.126367 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8db9df-aca7-454f-a48f-a7b0025cf71f None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:47.142520 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 50/97] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:31:46 2022] GET /identity/v3/role_assignments?user.id=c651e80ba10343f4ab505b7e2c378648&scope.project.id=5479a6e9aec54c4c9d3e119308cf2c88&role.id=83efac3b92eb44e783f4924c9d9ee842 => generated 259 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:31:49.012317 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90979fc-af99-4cc6-849a-1870ff0827fa None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:49.014007 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90979fc-af99-4cc6-849a-1870ff0827fa None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:49.014808 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90979fc-af99-4cc6-849a-1870ff0827fa None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:49.017876 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 48/98] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:31:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:31:49.024651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b3e78-af93-4616-a3ad-a6ce22212b48 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:49.024651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b3e78-af93-4616-a3ad-a6ce22212b48 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:49.025390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b3e78-af93-4616-a3ad-a6ce22212b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:49.054103 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-c04b3e78-af93-4616-a3ad-a6ce22212b48 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:31:49.055395 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-c04b3e78-af93-4616-a3ad-a6ce22212b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:31:49.101387 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:49.155224 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 51/99] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:31:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:31:49.750646 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90979fc-af99-4cc6-849a-1870ff0827fa None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:50.001145 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90979fc-af99-4cc6-849a-1870ff0827fa None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:50.075613 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f2b78-58f8-4df4-8c50-40ed3b4ce853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:50.076727 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f2b78-58f8-4df4-8c50-40ed3b4ce853 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:50.077225 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f2b78-58f8-4df4-8c50-40ed3b4ce853 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:50.077710 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f2b78-58f8-4df4-8c50-40ed3b4ce853 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:50.109347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614f2b78-58f8-4df4-8c50-40ed3b4ce853 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:31:50.124797 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-614f2b78-58f8-4df4-8c50-40ed3b4ce853 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:31:50.124797 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:31:50.124797 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:31:50.124797 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:31:50.124797 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:31:50.124797 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:31:50.124797 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:31:50.124797 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:31:50.124797 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:31:50.124797 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:31:50.124797 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:31:50.124797 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:31:50.124797 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:31:50.124797 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:31:50.124797 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:31:50.124797 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:31:50.125779 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:31:50.125779 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:31:50.125779 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:31:50.125779 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:31:50.125779 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:31:50.125779 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:31:50.125779 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:31:50.125779 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:31:50.125779 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:31:50.125779 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:31:50.125779 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:31:50.125779 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:31:50.125779 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:31:50.125779 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:31:50.125779 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:31:50.125779 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:31:50.126671 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:31:50.126671 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:31:50.126671 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:31:50.126671 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:31:50.126671 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:31:50.126671 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:31:50.126671 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:31:50.126671 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:31:50.126671 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:31:50.126671 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:31:50.126671 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:31:50.126671 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:31:50.128601 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 49/100] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Sep 22 02:31:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:31:50.138626 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04b3e78-af93-4616-a3ad-a6ce22212b48 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:50.265510 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0defebc-f26b-425c-b426-a44c6f4cbec1 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:50.265943 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0defebc-f26b-425c-b426-a44c6f4cbec1 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:50.266442 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0defebc-f26b-425c-b426-a44c6f4cbec1 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:50.269165 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 50/101] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:31:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:31:50.299845 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a129e55-d0f4-4d4f-ad97-d1783fc897b0 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:50.300820 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a129e55-d0f4-4d4f-ad97-d1783fc897b0 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:50.302448 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a129e55-d0f4-4d4f-ad97-d1783fc897b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:50.342534 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-5a129e55-d0f4-4d4f-ad97-d1783fc897b0 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:31:50.343851 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-5a129e55-d0f4-4d4f-ad97-d1783fc897b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:31:50.392620 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04b3e78-af93-4616-a3ad-a6ce22212b48 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:50.455396 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:50.469397 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd18f7a-3701-42cc-8a6d-315ff4efdd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:50.470042 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd18f7a-3701-42cc-8a6d-315ff4efdd7c None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:50.470338 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd18f7a-3701-42cc-8a6d-315ff4efdd7c None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:50.470784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd18f7a-3701-42cc-8a6d-315ff4efdd7c None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:50.471383 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd18f7a-3701-42cc-8a6d-315ff4efdd7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:31:50.471707 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd18f7a-3701-42cc-8a6d-315ff4efdd7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:50.474710 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd18f7a-3701-42cc-8a6d-315ff4efdd7c None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:50.491892 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 52/102] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:31:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:50.496317 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 51/103] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:31:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 02:31:50.501184 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd18f7a-3701-42cc-8a6d-315ff4efdd7c None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:50.547568 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:50.654695 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd18f7a-3701-42cc-8a6d-315ff4efdd7c None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:50.714308 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf55e6-cc63-45b6-bb54-13ddeee53406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:50.715561 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf55e6-cc63-45b6-bb54-13ddeee53406 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:50.716003 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf55e6-cc63-45b6-bb54-13ddeee53406 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:50.716441 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf55e6-cc63-45b6-bb54-13ddeee53406 None admin] PATH_INFO: `/v3/users/778bd65251774785bdd2460a6778044c` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:50.724276 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bf55e6-cc63-45b6-bb54-13ddeee53406 None admin] RBAC: Authorizing `identity:get_user(user_id=778bd65251774785bdd2460a6778044c)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:50.726859 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bf55e6-cc63-45b6-bb54-13ddeee53406 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:50.729628 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 53/104] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:31:50 2022] GET /identity/v3/users/778bd65251774785bdd2460a6778044c => generated 247 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:31:50.738654 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a129e55-d0f4-4d4f-ad97-d1783fc897b0 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:50.897673 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a129e55-d0f4-4d4f-ad97-d1783fc897b0 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:50.947187 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50936400-2249-4eb6-9c38-71d196b34df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:50.947858 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50936400-2249-4eb6-9c38-71d196b34df7 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:50.948152 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50936400-2249-4eb6-9c38-71d196b34df7 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:50.948442 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50936400-2249-4eb6-9c38-71d196b34df7 None admin] PATH_INFO: `/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:50.950348 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50936400-2249-4eb6-9c38-71d196b34df7 None admin] RBAC: Authorizing `identity:get_project(project_id=6a5cfaff6dc7495f9a0648c76c9f6eb7)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:50.952018 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50936400-2249-4eb6-9c38-71d196b34df7 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:50.953824 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 52/105] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:31:50 2022] GET /identity/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:31:50.962742 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf55e6-cc63-45b6-bb54-13ddeee53406 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:51.051219 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980ec77-3583-4d12-85eb-f46570571217 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:51.051643 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980ec77-3583-4d12-85eb-f46570571217 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:51.052070 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980ec77-3583-4d12-85eb-f46570571217 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:51.054251 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 53/106] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:31:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:31:51.064593 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b59613-43da-4d20-92fb-793c3a00299f None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:51.065016 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b59613-43da-4d20-92fb-793c3a00299f None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:51.065436 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b59613-43da-4d20-92fb-793c3a00299f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:51.144479 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-42b59613-43da-4d20-92fb-793c3a00299f None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:31:51.147460 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-42b59613-43da-4d20-92fb-793c3a00299f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:31:51.201090 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:51.207382 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf55e6-cc63-45b6-bb54-13ddeee53406 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:51.265852 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:51.287070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9862f7a-d3d5-426e-b182-9bb0e72b2a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:51.287785 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9862f7a-d3d5-426e-b182-9bb0e72b2a9c None admin] REQUEST_METHOD: `PUT` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:51.288076 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9862f7a-d3d5-426e-b182-9bb0e72b2a9c None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:51.288368 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9862f7a-d3d5-426e-b182-9bb0e72b2a9c None admin] PATH_INFO: `/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7/users/778bd65251774785bdd2460a6778044c/roles/31a687548f55493b8ea666fe7e350303` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:51.295725 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9862f7a-d3d5-426e-b182-9bb0e72b2a9c None admin] RBAC: Authorizing `identity:create_grant(project_id=6a5cfaff6dc7495f9a0648c76c9f6eb7, user_id=778bd65251774785bdd2460a6778044c, role_id=31a687548f55493b8ea666fe7e350303)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:51.298773 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9862f7a-d3d5-426e-b182-9bb0e72b2a9c None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:51.301735 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 54/107] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:31:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 02:31:51.303607 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b59613-43da-4d20-92fb-793c3a00299f None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:51.323324 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 54/108] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Sep 22 02:31:50 2022] PUT /identity/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7/users/778bd65251774785bdd2460a6778044c/roles/31a687548f55493b8ea666fe7e350303 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 02:31:51.325052 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365498f-dd2b-4138-b2e7-0cdc84ea41cb None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:51.325354 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365498f-dd2b-4138-b2e7-0cdc84ea41cb None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:51.325640 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365498f-dd2b-4138-b2e7-0cdc84ea41cb None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:51.327220 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 55/109] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:31:51.338736 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2d58a-8a77-4d8a-a672-ba509050d8ec None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:51.339106 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2d58a-8a77-4d8a-a672-ba509050d8ec None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:51.339349 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2d58a-8a77-4d8a-a672-ba509050d8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:51.380020 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-e7a2d58a-8a77-4d8a-a672-ba509050d8ec None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:31:51.381398 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-e7a2d58a-8a77-4d8a-a672-ba509050d8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:31:51.447303 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b59613-43da-4d20-92fb-793c3a00299f None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:51.561848 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affc7093-3059-4ec2-bc03-51bd23af92f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:51.563500 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc7093-3059-4ec2-bc03-51bd23af92f1 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:51.564335 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc7093-3059-4ec2-bc03-51bd23af92f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:51.565126 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc7093-3059-4ec2-bc03-51bd23af92f1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:51.586115 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affc7093-3059-4ec2-bc03-51bd23af92f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:31:51.596399 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-affc7093-3059-4ec2-bc03-51bd23af92f1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 02:31:51.596399 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:31:51.596399 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:31:51.596399 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:31:51.596399 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:31:51.596399 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:31:51.596399 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:31:51.596399 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:31:51.596399 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:31:51.596399 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:31:51.596399 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:31:51.596399 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:31:51.596399 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:31:51.596399 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:31:51.596399 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:31:51.596399 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:31:51.598169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:31:51.598169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:31:51.598169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:31:51.598169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:31:51.598169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:31:51.598169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:31:51.598169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:31:51.598169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:31:51.598169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:31:51.598169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:31:51.598169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:31:51.598169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:31:51.598169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:31:51.598169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:31:51.598169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:31:51.598169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:31:51.599677 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:31:51.599677 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:31:51.599677 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:31:51.599677 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:31:51.599677 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:31:51.599677 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:31:51.599677 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:31:51.599677 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:31:51.599677 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:31:51.599677 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:31:51.599677 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 02:31:51.599677 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:31:51.604528 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 55/110] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:31:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:31:51.608310 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f89a3-eb72-4d51-addf-6e16f0c28695 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:51.609073 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 56/111] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:31:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:31:51.609493 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f89a3-eb72-4d51-addf-6e16f0c28695 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:51.610205 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f89a3-eb72-4d51-addf-6e16f0c28695 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:51.615497 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 56/112] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:31:51 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:31:51.616422 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2d58a-8a77-4d8a-a672-ba509050d8ec None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:51.624083 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac140b11-c765-43cf-950c-25490a21a80b None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:51.624981 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac140b11-c765-43cf-950c-25490a21a80b None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:51.625664 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac140b11-c765-43cf-950c-25490a21a80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:51.689878 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-ac140b11-c765-43cf-950c-25490a21a80b None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:31:51.692593 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-ac140b11-c765-43cf-950c-25490a21a80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:31:51.764357 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:51.801046 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:51.832445 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2d58a-8a77-4d8a-a672-ba509050d8ec None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:51.843182 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 57/113] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:31:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:31:51.845100 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac140b11-c765-43cf-950c-25490a21a80b None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:51.940991 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0f4990-bf0a-42b0-9460-4435c2f67e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:51.942032 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f4990-bf0a-42b0-9460-4435c2f67e62 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:51.942531 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f4990-bf0a-42b0-9460-4435c2f67e62 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:51.943045 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f4990-bf0a-42b0-9460-4435c2f67e62 None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:51.947620 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0f4990-bf0a-42b0-9460-4435c2f67e62 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:31:51.948189 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0f4990-bf0a-42b0-9460-4435c2f67e62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:51.950849 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0f4990-bf0a-42b0-9460-4435c2f67e62 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:51.966729 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 57/114] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Sep 22 02:31:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:51.969113 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806342d7-fc1b-47ce-953a-7f872daecdfb None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:51.969620 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806342d7-fc1b-47ce-953a-7f872daecdfb None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:51.970119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806342d7-fc1b-47ce-953a-7f872daecdfb None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:51.972900 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 58/115] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:31:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:31:51.974204 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806342d7-fc1b-47ce-953a-7f872daecdfb None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:52.079000 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac140b11-c765-43cf-950c-25490a21a80b None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:52.162597 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583ec740-59c2-46cc-88cf-8078c65a563e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:52.163933 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ec740-59c2-46cc-88cf-8078c65a563e None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:52.164581 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ec740-59c2-46cc-88cf-8078c65a563e None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:52.165085 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ec740-59c2-46cc-88cf-8078c65a563e None admin] PATH_INFO: `/v3/domains/default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:52.167238 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583ec740-59c2-46cc-88cf-8078c65a563e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:52.171190 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583ec740-59c2-46cc-88cf-8078c65a563e None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:52.174869 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 58/116] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:31:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:52.176944 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c70df1-162b-4b35-9225-494b4170eddd None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:52.177708 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c70df1-162b-4b35-9225-494b4170eddd None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:52.178217 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c70df1-162b-4b35-9225-494b4170eddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:52.216031 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-58c70df1-162b-4b35-9225-494b4170eddd None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:31:52.218706 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-58c70df1-162b-4b35-9225-494b4170eddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:31:52.258786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:52.263045 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806342d7-fc1b-47ce-953a-7f872daecdfb None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:52.319794 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06256737-89b9-4b64-bb06-1bc990055a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:52.320467 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06256737-89b9-4b64-bb06-1bc990055a05 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:52.320821 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06256737-89b9-4b64-bb06-1bc990055a05 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:52.321099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06256737-89b9-4b64-bb06-1bc990055a05 None admin] PATH_INFO: `/v3/users/c4029608b5a342f78d43dabc3c33197a` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:52.322941 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 59/117] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:31:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:31:52.332161 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c70df1-162b-4b35-9225-494b4170eddd None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:52.341191 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06256737-89b9-4b64-bb06-1bc990055a05 None admin] RBAC: Authorizing `identity:get_user(user_id=c4029608b5a342f78d43dabc3c33197a)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:52.345194 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06256737-89b9-4b64-bb06-1bc990055a05 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:52.359170 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 59/118] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:31:51 2022] GET /identity/v3/users/c4029608b5a342f78d43dabc3c33197a => generated 289 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:52.359170 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06256737-89b9-4b64-bb06-1bc990055a05 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:52.374021 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:52.509112 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06256737-89b9-4b64-bb06-1bc990055a05 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:52.579476 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cac117-6e55-47e1-9881-181a0a67bf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:52.580594 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cac117-6e55-47e1-9881-181a0a67bf89 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:52.581157 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cac117-6e55-47e1-9881-181a0a67bf89 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:52.581705 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cac117-6e55-47e1-9881-181a0a67bf89 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:52.588616 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c70df1-162b-4b35-9225-494b4170eddd None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:52.602303 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cac117-6e55-47e1-9881-181a0a67bf89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:31:52.613450 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-02cac117-6e55-47e1-9881-181a0a67bf89 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 02:31:52.613450 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:31:52.613450 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:31:52.613450 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:31:52.613450 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:31:52.613450 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:31:52.613450 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:31:52.613450 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:31:52.613450 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:31:52.613450 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:31:52.613450 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:31:52.613450 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:31:52.613450 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:31:52.613450 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:31:52.613450 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:31:52.613450 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:31:52.615476 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:31:52.615476 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:31:52.615476 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:31:52.615476 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:31:52.615476 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:31:52.615476 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:31:52.615476 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:31:52.615476 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:31:52.615476 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:31:52.615476 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:31:52.615476 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:31:52.615476 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:31:52.615476 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:31:52.615476 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:31:52.615476 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:31:52.615476 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:31:52.617267 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:31:52.617267 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:31:52.617267 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:31:52.617267 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:31:52.617267 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:31:52.617267 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:31:52.617267 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:31:52.617267 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:31:52.617267 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:31:52.617267 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:31:52.617267 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 02:31:52.617267 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:31:52.618543 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 60/119] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Sep 22 02:31:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:31:52.621938 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cac117-6e55-47e1-9881-181a0a67bf89 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:52.665737 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e87405-085f-48d6-af05-10d1b8ea73e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:52.666957 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e87405-085f-48d6-af05-10d1b8ea73e9 None admin] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:52.667514 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e87405-085f-48d6-af05-10d1b8ea73e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:52.668079 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e87405-085f-48d6-af05-10d1b8ea73e9 None admin] PATH_INFO: `/v3/projects` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:52.670051 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e87405-085f-48d6-af05-10d1b8ea73e9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:52.673113 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e87405-085f-48d6-af05-10d1b8ea73e9 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:52.724250 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 60/120] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Sep 22 02:31:52 2022] POST /identity/v3/projects => generated 302 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:31:52.725954 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e87405-085f-48d6-af05-10d1b8ea73e9 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:52.771774 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:52.784395 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:52.860343 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cac117-6e55-47e1-9881-181a0a67bf89 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:52.914133 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e87405-085f-48d6-af05-10d1b8ea73e9 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:52.961698 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e1884-936c-43e6-b11a-940a479a42ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:52.962346 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e1884-936c-43e6-b11a-940a479a42ef None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:52.962630 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e1884-936c-43e6-b11a-940a479a42ef None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:52.962940 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e1884-936c-43e6-b11a-940a479a42ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:52.973274 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2fee1-5e0a-46c6-9c53-a126abfd87a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:52.973957 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2fee1-5e0a-46c6-9c53-a126abfd87a4 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:52.974245 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2fee1-5e0a-46c6-9c53-a126abfd87a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:52.975569 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2fee1-5e0a-46c6-9c53-a126abfd87a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:52.999940 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc2fee1-5e0a-46c6-9c53-a126abfd87a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:31:53.006230 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-2bc2fee1-5e0a-46c6-9c53-a126abfd87a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 02:31:53.006230 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:31:53.006230 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:31:53.006230 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:31:53.006230 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:31:53.006230 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:31:53.006230 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:31:53.006230 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:31:53.006230 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:31:53.006230 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:31:53.006230 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:31:53.006230 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:31:53.006230 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:31:53.006230 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:31:53.006230 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:31:53.006230 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:31:53.009524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:31:53.009524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:31:53.009524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:31:53.009524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:31:53.009524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:31:53.009524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:31:53.009524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:31:53.009524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:31:53.009524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:31:53.009524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:31:53.009524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:31:53.009524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:31:53.009524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:31:53.009524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:31:53.009524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:31:53.009524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:31:53.011080 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:31:53.011080 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:31:53.011080 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:31:53.011080 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:31:53.011080 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:31:53.011080 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:31:53.011080 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:31:53.011080 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:31:53.011080 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:31:53.011080 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:31:53.011080 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 02:31:53.011080 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:31:53.011080 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 61/121] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:31:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:31:53.011080 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2fee1-5e0a-46c6-9c53-a126abfd87a4 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:53.062507 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e1884-936c-43e6-b11a-940a479a42ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:53.064100 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e1884-936c-43e6-b11a-940a479a42ef None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:53.113286 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:53.140035 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2fee1-5e0a-46c6-9c53-a126abfd87a4 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:53.170980 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:53.211155 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3c6cc-f5b6-4efe-89a3-e2ab5a680a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:53.213773 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3c6cc-f5b6-4efe-89a3-e2ab5a680a5d None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:53.214996 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3c6cc-f5b6-4efe-89a3-e2ab5a680a5d None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:53.216398 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3c6cc-f5b6-4efe-89a3-e2ab5a680a5d None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:53.217677 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e3c6cc-f5b6-4efe-89a3-e2ab5a680a5d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:31:53.218690 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e3c6cc-f5b6-4efe-89a3-e2ab5a680a5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:53.220660 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e3c6cc-f5b6-4efe-89a3-e2ab5a680a5d None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:53.228951 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 61/122] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:31:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 02:31:53.232047 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e1884-936c-43e6-b11a-940a479a42ef None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:53.239063 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 62/123] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:31:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:53.240312 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3c6cc-f5b6-4efe-89a3-e2ab5a680a5d None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:53.367255 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e1884-936c-43e6-b11a-940a479a42ef None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:53.386061 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3c6cc-f5b6-4efe-89a3-e2ab5a680a5d None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:53.419160 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f3673c-5a6f-41f4-9662-51321b29c708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:53.420377 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3673c-5a6f-41f4-9662-51321b29c708 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:53.420818 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3673c-5a6f-41f4-9662-51321b29c708 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:53.421449 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3673c-5a6f-41f4-9662-51321b29c708 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:53.427358 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3833d19d-6271-4af0-9e1c-90c3de5eaae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:53.428086 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833d19d-6271-4af0-9e1c-90c3de5eaae9 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:53.428443 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833d19d-6271-4af0-9e1c-90c3de5eaae9 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:53.428793 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833d19d-6271-4af0-9e1c-90c3de5eaae9 None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:53.429560 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3833d19d-6271-4af0-9e1c-90c3de5eaae9 None admin] RBAC: Adding query filter params (name=member) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:31:53.429949 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3833d19d-6271-4af0-9e1c-90c3de5eaae9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:53.431497 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3833d19d-6271-4af0-9e1c-90c3de5eaae9 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:53.436989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f3673c-5a6f-41f4-9662-51321b29c708 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:31:53.441579 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 63/124] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Sep 22 02:31:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:53.442370 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3833d19d-6271-4af0-9e1c-90c3de5eaae9 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:53.444126 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-b6f3673c-5a6f-41f4-9662-51321b29c708 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 02:31:53.444126 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:31:53.444126 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:31:53.444126 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:31:53.444126 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:31:53.444126 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:31:53.444126 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:31:53.444126 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:31:53.444126 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:31:53.444126 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:31:53.444126 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:31:53.444126 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:31:53.444126 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:31:53.444126 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:31:53.444126 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:31:53.444126 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:31:53.445241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:31:53.445241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:31:53.445241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:31:53.445241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:31:53.445241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:31:53.445241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:31:53.445241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:31:53.445241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:31:53.445241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:31:53.445241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:31:53.445241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:31:53.445241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:31:53.445241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:31:53.445241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:31:53.445241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:31:53.445241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:31:53.446236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:31:53.446236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:31:53.446236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:31:53.446236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:31:53.446236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:31:53.446236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:31:53.446236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:31:53.446236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:31:53.446236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:31:53.446236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:31:53.446236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 02:31:53.446236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:31:53.448677 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 62/125] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:31:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:31:53.448942 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:53.451660 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f3673c-5a6f-41f4-9662-51321b29c708 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:53.492630 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:53.591720 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f3673c-5a6f-41f4-9662-51321b29c708 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:53.596726 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3833d19d-6271-4af0-9e1c-90c3de5eaae9 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:53.646184 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136df2b2-c794-4198-95f5-00bb0e505e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:53.646972 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136df2b2-c794-4198-95f5-00bb0e505e2c None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:53.647367 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136df2b2-c794-4198-95f5-00bb0e505e2c None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:53.647750 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136df2b2-c794-4198-95f5-00bb0e505e2c None admin] PATH_INFO: `/v3/users/c651e80ba10343f4ab505b7e2c378648` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:53.649266 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136df2b2-c794-4198-95f5-00bb0e505e2c None admin] RBAC: Authorizing `identity:get_user(user_id=c651e80ba10343f4ab505b7e2c378648)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:53.651274 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136df2b2-c794-4198-95f5-00bb0e505e2c None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:53.653816 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 63/126] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:31:53 2022] GET /identity/v3/users/c651e80ba10343f4ab505b7e2c378648 => generated 275 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:53.654934 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136df2b2-c794-4198-95f5-00bb0e505e2c None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:53.664099 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80bae5a-9742-4720-ade7-19c8b7dd164c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:53.666090 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80bae5a-9742-4720-ade7-19c8b7dd164c None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:53.666605 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80bae5a-9742-4720-ade7-19c8b7dd164c None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:53.667159 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80bae5a-9742-4720-ade7-19c8b7dd164c None admin] PATH_INFO: `/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:53.671128 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80bae5a-9742-4720-ade7-19c8b7dd164c None admin] RBAC: Authorizing `identity:get_project(project_id=6a5cfaff6dc7495f9a0648c76c9f6eb7)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:53.674507 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80bae5a-9742-4720-ade7-19c8b7dd164c None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:53.678456 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 64/127] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:31:53 2022] GET /identity/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7 => generated 298 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:53.680095 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80bae5a-9742-4720-ade7-19c8b7dd164c None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:53.786350 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:53.794937 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136df2b2-c794-4198-95f5-00bb0e505e2c None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:53.841117 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d446f7-68d0-450d-bd88-25880bc740de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:53.842614 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d446f7-68d0-450d-bd88-25880bc740de None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:53.843502 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d446f7-68d0-450d-bd88-25880bc740de None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:53.844557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d446f7-68d0-450d-bd88-25880bc740de None admin] PATH_INFO: `/v3/users/c651e80ba10343f4ab505b7e2c378648` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:53.847663 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:53.849088 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d446f7-68d0-450d-bd88-25880bc740de None admin] RBAC: Authorizing `identity:get_user(user_id=c651e80ba10343f4ab505b7e2c378648)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:53.852051 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d446f7-68d0-450d-bd88-25880bc740de None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:53.862794 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 64/128] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:31:53 2022] GET /identity/v3/users/c651e80ba10343f4ab505b7e2c378648 => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:53.862794 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d446f7-68d0-450d-bd88-25880bc740de None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:53.908993 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80bae5a-9742-4720-ade7-19c8b7dd164c None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:53.960620 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d862f4-f3e5-4628-bd49-94ec2c7f19a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:53.961631 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d862f4-f3e5-4628-bd49-94ec2c7f19a3 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:53.962127 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d862f4-f3e5-4628-bd49-94ec2c7f19a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:53.962760 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d862f4-f3e5-4628-bd49-94ec2c7f19a3 None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:53.963689 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d862f4-f3e5-4628-bd49-94ec2c7f19a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:31:53.964560 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d862f4-f3e5-4628-bd49-94ec2c7f19a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:53.966507 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d862f4-f3e5-4628-bd49-94ec2c7f19a3 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:53.977521 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 65/129] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Sep 22 02:31:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:53.979331 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d862f4-f3e5-4628-bd49-94ec2c7f19a3 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:54.049231 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d446f7-68d0-450d-bd88-25880bc740de None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:54.109379 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d862f4-f3e5-4628-bd49-94ec2c7f19a3 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:54.116042 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0da464-cab3-4895-b0a5-6cd13420b260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:54.117403 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0da464-cab3-4895-b0a5-6cd13420b260 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:54.118510 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0da464-cab3-4895-b0a5-6cd13420b260 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:54.118695 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0da464-cab3-4895-b0a5-6cd13420b260 None admin] PATH_INFO: `/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:54.120964 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0da464-cab3-4895-b0a5-6cd13420b260 None admin] RBAC: Authorizing `identity:get_project(project_id=6a5cfaff6dc7495f9a0648c76c9f6eb7)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:54.123695 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0da464-cab3-4895-b0a5-6cd13420b260 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:54.127146 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 65/130] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:31:53 2022] GET /identity/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7 => generated 298 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:54.128661 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0da464-cab3-4895-b0a5-6cd13420b260 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:54.152139 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957dd9e7-c797-4391-9254-28d28ab3f7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:54.153067 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957dd9e7-c797-4391-9254-28d28ab3f7de None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:54.153793 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957dd9e7-c797-4391-9254-28d28ab3f7de None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:54.154217 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957dd9e7-c797-4391-9254-28d28ab3f7de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:54.155161 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957dd9e7-c797-4391-9254-28d28ab3f7de None admin] RBAC: Adding query filter params (role.id=13d7b9e519274c8bb245c8218571dee8, scope.project.id=6a5cfaff6dc7495f9a0648c76c9f6eb7, user.id=c4029608b5a342f78d43dabc3c33197a) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:31:54.155616 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957dd9e7-c797-4391-9254-28d28ab3f7de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:54.157463 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957dd9e7-c797-4391-9254-28d28ab3f7de None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:54.160669 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:54.170427 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 66/131] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:31:53 2022] GET /identity/v3/role_assignments?user.id=c4029608b5a342f78d43dabc3c33197a&scope.project.id=6a5cfaff6dc7495f9a0648c76c9f6eb7&role.id=13d7b9e519274c8bb245c8218571dee8 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:54.171531 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957dd9e7-c797-4391-9254-28d28ab3f7de None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:54.207431 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:54.348983 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0da464-cab3-4895-b0a5-6cd13420b260 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:54.416584 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b773a0-0ae7-4cd9-941e-fec1b175c32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:54.417565 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b773a0-0ae7-4cd9-941e-fec1b175c32e None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:54.417565 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b773a0-0ae7-4cd9-941e-fec1b175c32e None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:54.417780 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b773a0-0ae7-4cd9-941e-fec1b175c32e None admin] PATH_INFO: `/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:54.419545 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b773a0-0ae7-4cd9-941e-fec1b175c32e None admin] RBAC: Authorizing `identity:get_project(project_id=6a5cfaff6dc7495f9a0648c76c9f6eb7)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:54.421193 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b773a0-0ae7-4cd9-941e-fec1b175c32e None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:54.423100 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 66/132] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:31:54 2022] GET /identity/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7 => generated 298 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:54.424178 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b773a0-0ae7-4cd9-941e-fec1b175c32e None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:54.440200 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957dd9e7-c797-4391-9254-28d28ab3f7de None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:54.492092 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d72b2-2fbc-457f-839b-f020478f5288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:54.493116 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d72b2-2fbc-457f-839b-f020478f5288 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:54.493631 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d72b2-2fbc-457f-839b-f020478f5288 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:54.494107 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d72b2-2fbc-457f-839b-f020478f5288 None admin] PATH_INFO: `/v3/users/c651e80ba10343f4ab505b7e2c378648` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:54.495565 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d72b2-2fbc-457f-839b-f020478f5288 None admin] RBAC: Authorizing `identity:get_user(user_id=c651e80ba10343f4ab505b7e2c378648)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:54.497328 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d72b2-2fbc-457f-839b-f020478f5288 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:54.499234 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 67/133] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:31:54 2022] GET /identity/v3/users/c651e80ba10343f4ab505b7e2c378648 => generated 275 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:54.500772 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd61c80-46b1-4405-a8e5-235233cebb12 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:54.501278 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd61c80-46b1-4405-a8e5-235233cebb12 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:54.501771 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd61c80-46b1-4405-a8e5-235233cebb12 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:54.503493 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 68/134] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:31:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:31:54.504511 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd61c80-46b1-4405-a8e5-235233cebb12 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:54.554286 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b773a0-0ae7-4cd9-941e-fec1b175c32e None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:54.558177 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:54.581311 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:54.598096 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd347f4a-8157-46b9-8b1c-8847e2fbb3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:54.598789 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd347f4a-8157-46b9-8b1c-8847e2fbb3a4 None admin] REQUEST_METHOD: `PUT` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:54.599111 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd347f4a-8157-46b9-8b1c-8847e2fbb3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:54.599397 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd347f4a-8157-46b9-8b1c-8847e2fbb3a4 None admin] PATH_INFO: `/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7/users/c651e80ba10343f4ab505b7e2c378648/roles/adec3004b56243fa923bd9f0082a2728` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:54.602191 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd347f4a-8157-46b9-8b1c-8847e2fbb3a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a5cfaff6dc7495f9a0648c76c9f6eb7, user_id=c651e80ba10343f4ab505b7e2c378648, role_id=adec3004b56243fa923bd9f0082a2728)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:54.603860 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd347f4a-8157-46b9-8b1c-8847e2fbb3a4 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:54.627168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 67/135] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Sep 22 02:31:54 2022] PUT /identity/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7/users/c651e80ba10343f4ab505b7e2c378648/roles/adec3004b56243fa923bd9f0082a2728 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 02:31:54.628200 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd347f4a-8157-46b9-8b1c-8847e2fbb3a4 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:54.630995 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd61c80-46b1-4405-a8e5-235233cebb12 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:54.671945 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61406734-e942-4a1e-bb52-20583bbacf21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:54.673024 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61406734-e942-4a1e-bb52-20583bbacf21 None admin] REQUEST_METHOD: `PUT` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:54.673542 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61406734-e942-4a1e-bb52-20583bbacf21 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:54.674036 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61406734-e942-4a1e-bb52-20583bbacf21 None admin] PATH_INFO: `/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7/users/c651e80ba10343f4ab505b7e2c378648/roles/83efac3b92eb44e783f4924c9d9ee842` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:54.676480 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61406734-e942-4a1e-bb52-20583bbacf21 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a5cfaff6dc7495f9a0648c76c9f6eb7, user_id=c651e80ba10343f4ab505b7e2c378648, role_id=83efac3b92eb44e783f4924c9d9ee842)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:54.678439 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61406734-e942-4a1e-bb52-20583bbacf21 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:54.697299 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 69/136] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Sep 22 02:31:54 2022] PUT /identity/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7/users/c651e80ba10343f4ab505b7e2c378648/roles/83efac3b92eb44e783f4924c9d9ee842 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 02:31:54.699828 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d757b1-9ebf-4ec3-b50b-19eb43fff4c7 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:54.700443 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d757b1-9ebf-4ec3-b50b-19eb43fff4c7 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:54.701033 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d757b1-9ebf-4ec3-b50b-19eb43fff4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:54.740869 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-c2d757b1-9ebf-4ec3-b50b-19eb43fff4c7 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:31:54.742553 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-c2d757b1-9ebf-4ec3-b50b-19eb43fff4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:31:54.835625 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 70/137] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:31:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:31:54.845231 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd347f4a-8157-46b9-8b1c-8847e2fbb3a4 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:54.907792 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10cee48-fdc5-43ff-8edd-0ec939528eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:54.908490 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10cee48-fdc5-43ff-8edd-0ec939528eff None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:54.908784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10cee48-fdc5-43ff-8edd-0ec939528eff None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:54.909073 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10cee48-fdc5-43ff-8edd-0ec939528eff None admin] PATH_INFO: `/v3/projects/5479a6e9aec54c4c9d3e119308cf2c88` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:54.910583 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10cee48-fdc5-43ff-8edd-0ec939528eff None admin] RBAC: Authorizing `identity:get_project(project_id=5479a6e9aec54c4c9d3e119308cf2c88)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:54.912175 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10cee48-fdc5-43ff-8edd-0ec939528eff None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:54.914101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 68/138] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:31:54 2022] GET /identity/v3/projects/5479a6e9aec54c4c9d3e119308cf2c88 => generated 312 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:54.923605 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d757b1-9ebf-4ec3-b50b-19eb43fff4c7 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:55.008399 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:55.087681 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d757b1-9ebf-4ec3-b50b-19eb43fff4c7 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:55.129544 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a08e2-b121-416f-934c-d58d7f1da464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:55.130242 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a08e2-b121-416f-934c-d58d7f1da464 None admin] REQUEST_METHOD: `PUT` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:55.130531 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a08e2-b121-416f-934c-d58d7f1da464 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:55.130836 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a08e2-b121-416f-934c-d58d7f1da464 None admin] PATH_INFO: `/v3/projects/5479a6e9aec54c4c9d3e119308cf2c88/users/c651e80ba10343f4ab505b7e2c378648/roles/83efac3b92eb44e783f4924c9d9ee842` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:55.132933 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a08e2-b121-416f-934c-d58d7f1da464 None admin] RBAC: Authorizing `identity:create_grant(project_id=5479a6e9aec54c4c9d3e119308cf2c88, user_id=c651e80ba10343f4ab505b7e2c378648, role_id=83efac3b92eb44e783f4924c9d9ee842)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:55.134498 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a08e2-b121-416f-934c-d58d7f1da464 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:55.153139 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 71/139] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Sep 22 02:31:54 2022] PUT /identity/v3/projects/5479a6e9aec54c4c9d3e119308cf2c88/users/c651e80ba10343f4ab505b7e2c378648/roles/83efac3b92eb44e783f4924c9d9ee842 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 02:31:55.216089 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10cee48-fdc5-43ff-8edd-0ec939528eff None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:55.264993 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:55.398258 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10cee48-fdc5-43ff-8edd-0ec939528eff None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:55.450647 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598702b-e9f8-43f5-8479-e1bb903f0c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:55.451535 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598702b-e9f8-43f5-8479-e1bb903f0c82 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:55.451750 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598702b-e9f8-43f5-8479-e1bb903f0c82 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:55.452045 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598702b-e9f8-43f5-8479-e1bb903f0c82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:55.465663 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598702b-e9f8-43f5-8479-e1bb903f0c82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:31:55.471594 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-e598702b-e9f8-43f5-8479-e1bb903f0c82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:31:55.471594 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:31:55.471594 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:31:55.471594 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:31:55.471594 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:31:55.471594 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:31:55.471594 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:31:55.471594 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:31:55.471594 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:31:55.471594 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:31:55.471594 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:31:55.471594 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:31:55.471594 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:31:55.471594 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:31:55.471594 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:31:55.471594 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:31:55.472756 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:31:55.472756 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:31:55.472756 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:31:55.472756 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:31:55.472756 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:31:55.472756 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:31:55.472756 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:31:55.472756 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:31:55.472756 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:31:55.472756 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:31:55.472756 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:31:55.472756 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:31:55.472756 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:31:55.472756 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:31:55.472756 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:31:55.472756 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:31:55.473716 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:31:55.473716 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:31:55.473716 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:31:55.473716 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:31:55.473716 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:31:55.473716 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:31:55.473716 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:31:55.473716 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:31:55.473716 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:31:55.473716 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:31:55.473716 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:31:55.473716 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:31:55.473716 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 69/140] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Sep 22 02:31:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:31:55.479623 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a08e2-b121-416f-934c-d58d7f1da464 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:55.615378 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a08e2-b121-416f-934c-d58d7f1da464 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:55.637850 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:55.660286 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadbc16b-8875-4a1c-92f3-84885c7eba27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:55.660923 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadbc16b-8875-4a1c-92f3-84885c7eba27 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:55.661214 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadbc16b-8875-4a1c-92f3-84885c7eba27 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:55.661497 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadbc16b-8875-4a1c-92f3-84885c7eba27 None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:55.662244 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadbc16b-8875-4a1c-92f3-84885c7eba27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:31:55.662560 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadbc16b-8875-4a1c-92f3-84885c7eba27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:55.664167 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadbc16b-8875-4a1c-92f3-84885c7eba27 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:55.678139 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 72/141] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:31:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:31:55.684766 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598702b-e9f8-43f5-8479-e1bb903f0c82 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:55.821096 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:55.837211 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598702b-e9f8-43f5-8479-e1bb903f0c82 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:55.901122 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6264344d-6fa3-4d86-9940-aae33ffe2cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:55.901796 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264344d-6fa3-4d86-9940-aae33ffe2cd2 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:55.902097 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264344d-6fa3-4d86-9940-aae33ffe2cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:55.902926 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264344d-6fa3-4d86-9940-aae33ffe2cd2 None admin] PATH_INFO: `/v3/users/778bd65251774785bdd2460a6778044c` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:55.906943 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6264344d-6fa3-4d86-9940-aae33ffe2cd2 None admin] RBAC: Authorizing `identity:get_user(user_id=778bd65251774785bdd2460a6778044c)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:55.908415 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6264344d-6fa3-4d86-9940-aae33ffe2cd2 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:55.910515 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 70/142] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:31:55 2022] GET /identity/v3/users/778bd65251774785bdd2460a6778044c => generated 247 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:55.916573 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadbc16b-8875-4a1c-92f3-84885c7eba27 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:55.984072 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c3145-18e6-47dd-affa-921c7b0589a1 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:55.984432 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c3145-18e6-47dd-affa-921c7b0589a1 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:55.984690 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c3145-18e6-47dd-affa-921c7b0589a1 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:55.986439 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 71/143] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:31:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:31:56.007311 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f13bf7f-0ffb-4171-a680-06bd7a8370e7 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:56.007641 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f13bf7f-0ffb-4171-a680-06bd7a8370e7 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:56.007868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f13bf7f-0ffb-4171-a680-06bd7a8370e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:56.048060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-5f13bf7f-0ffb-4171-a680-06bd7a8370e7 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:31:56.049322 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-5f13bf7f-0ffb-4171-a680-06bd7a8370e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:31:56.135813 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadbc16b-8875-4a1c-92f3-84885c7eba27 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:56.189825 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 72/144] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:31:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:31:56.199696 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e81722-e633-4eaa-b031-fbd584c9032d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:56.200699 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e81722-e633-4eaa-b031-fbd584c9032d None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:56.201199 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e81722-e633-4eaa-b031-fbd584c9032d None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:56.201700 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e81722-e633-4eaa-b031-fbd584c9032d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:56.271865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:56.303803 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e81722-e633-4eaa-b031-fbd584c9032d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:56.305672 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e81722-e633-4eaa-b031-fbd584c9032d None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:56.442274 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 73/145] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:31:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 02:31:56.447700 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f13bf7f-0ffb-4171-a680-06bd7a8370e7 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:56.580379 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:56.602428 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f13bf7f-0ffb-4171-a680-06bd7a8370e7 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:56.651205 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba8d4e-7aa7-49a6-a877-484969341bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:56.651869 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba8d4e-7aa7-49a6-a877-484969341bb5 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:56.652151 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba8d4e-7aa7-49a6-a877-484969341bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:56.652953 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba8d4e-7aa7-49a6-a877-484969341bb5 None admin] PATH_INFO: `/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:56.654183 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba8d4e-7aa7-49a6-a877-484969341bb5 None admin] RBAC: Authorizing `identity:get_project(project_id=6a5cfaff6dc7495f9a0648c76c9f6eb7)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:56.655714 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba8d4e-7aa7-49a6-a877-484969341bb5 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:56.657899 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 73/146] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:31:56 2022] GET /identity/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:56.664388 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e81722-e633-4eaa-b031-fbd584c9032d None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:56.760494 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba8d4e-7aa7-49a6-a877-484969341bb5 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:56.792808 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:56.826897 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e81722-e633-4eaa-b031-fbd584c9032d None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:56.871331 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c4b0b-dd76-4acf-afa2-9eb38208b2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:56.871982 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c4b0b-dd76-4acf-afa2-9eb38208b2b9 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:56.872268 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c4b0b-dd76-4acf-afa2-9eb38208b2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:56.872546 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c4b0b-dd76-4acf-afa2-9eb38208b2b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:56.873322 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c4b0b-dd76-4acf-afa2-9eb38208b2b9 None admin] RBAC: Adding query filter params (role.id=31a687548f55493b8ea666fe7e350303, scope.project.id=6a5cfaff6dc7495f9a0648c76c9f6eb7, user.id=778bd65251774785bdd2460a6778044c) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:31:56.873636 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c4b0b-dd76-4acf-afa2-9eb38208b2b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:56.875997 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c4b0b-dd76-4acf-afa2-9eb38208b2b9 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:56.884176 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 74/147] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:31:56 2022] GET /identity/v3/role_assignments?user.id=778bd65251774785bdd2460a6778044c&scope.project.id=6a5cfaff6dc7495f9a0648c76c9f6eb7&role.id=31a687548f55493b8ea666fe7e350303 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:56.961313 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba8d4e-7aa7-49a6-a877-484969341bb5 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:57.011281 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6fcae7-2b0b-4093-bb99-f18fd4848950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:57.012806 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6fcae7-2b0b-4093-bb99-f18fd4848950 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:57.013357 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6fcae7-2b0b-4093-bb99-f18fd4848950 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:57.013866 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6fcae7-2b0b-4093-bb99-f18fd4848950 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:57.015602 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6fcae7-2b0b-4093-bb99-f18fd4848950 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:57.017349 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6fcae7-2b0b-4093-bb99-f18fd4848950 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:57.019289 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 74/148] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:31:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:31:57.031147 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c4b0b-dd76-4acf-afa2-9eb38208b2b9 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:57.184524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c4b0b-dd76-4acf-afa2-9eb38208b2b9 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:57.225414 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949c021-ba97-47f2-b8dd-b2d6070261b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:57.226071 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949c021-ba97-47f2-b8dd-b2d6070261b0 None admin] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:57.226358 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949c021-ba97-47f2-b8dd-b2d6070261b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:57.226637 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949c021-ba97-47f2-b8dd-b2d6070261b0 None admin] PATH_INFO: `/v3/users` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:57.227361 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e949c021-ba97-47f2-b8dd-b2d6070261b0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:57.228871 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e949c021-ba97-47f2-b8dd-b2d6070261b0 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:57.261020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:57.267483 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 75/149] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Sep 22 02:31:57 2022] POST /identity/v3/users => generated 283 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:31:58.141911 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a6813-8707-4c18-80b0-5043499c9895 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:58.141911 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a6813-8707-4c18-80b0-5043499c9895 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:58.141911 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a6813-8707-4c18-80b0-5043499c9895 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:58.143127 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 75/150] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:31:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:31:58.149602 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c1e07-d30c-4f5c-8bb1-8a63b77d8b06 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:58.150111 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c1e07-d30c-4f5c-8bb1-8a63b77d8b06 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:58.150459 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c1e07-d30c-4f5c-8bb1-8a63b77d8b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:58.172273 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-d92c1e07-d30c-4f5c-8bb1-8a63b77d8b06 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:31:58.173537 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-d92c1e07-d30c-4f5c-8bb1-8a63b77d8b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:31:58.262451 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 76/151] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:31:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 02:31:58.572618 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a6813-8707-4c18-80b0-5043499c9895 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:58.611719 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:58.699458 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a5ac1-a97d-42fd-8d19-77e23c06ff7d None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:58.699862 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a5ac1-a97d-42fd-8d19-77e23c06ff7d None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:58.700003 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a5ac1-a97d-42fd-8d19-77e23c06ff7d None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:58.701672 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 77/152] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:31:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:31:58.708108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd482e5-ba63-4ae8-a245-599d954d8a5a None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:58.708618 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd482e5-ba63-4ae8-a245-599d954d8a5a None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:58.708908 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd482e5-ba63-4ae8-a245-599d954d8a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:58.734933 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-dbd482e5-ba63-4ae8-a245-599d954d8a5a None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:31:58.739057 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-dbd482e5-ba63-4ae8-a245-599d954d8a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:31:58.785219 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a6813-8707-4c18-80b0-5043499c9895 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:58.867942 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc3821e-f85e-49b9-9245-1c3d3cbe80f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:58.868475 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3821e-f85e-49b9-9245-1c3d3cbe80f6 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:58.868766 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3821e-f85e-49b9-9245-1c3d3cbe80f6 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:58.869041 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3821e-f85e-49b9-9245-1c3d3cbe80f6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:58.873327 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 78/153] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:31:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 02:31:58.875927 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f51d1-dbab-43e1-a450-7b16e6c604c3 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:58.876677 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f51d1-dbab-43e1-a450-7b16e6c604c3 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:58.877541 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f51d1-dbab-43e1-a450-7b16e6c604c3 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:58.880295 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 79/154] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:31:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:31:58.888578 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc3821e-f85e-49b9-9245-1c3d3cbe80f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:31:58.890460 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d6035-6484-40d1-a7ad-6b37d25f36ac None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:58.891259 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d6035-6484-40d1-a7ad-6b37d25f36ac None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:58.891970 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d6035-6484-40d1-a7ad-6b37d25f36ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:58.893681 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-ccc3821e-f85e-49b9-9245-1c3d3cbe80f6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 02:31:58.893681 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:31:58.893681 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:31:58.893681 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:31:58.893681 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:31:58.893681 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:31:58.893681 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:31:58.893681 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:31:58.893681 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:31:58.893681 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:31:58.893681 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:31:58.893681 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:31:58.893681 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:31:58.893681 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:31:58.893681 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:31:58.893681 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:31:58.895253 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:31:58.895253 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:31:58.895253 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:31:58.895253 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:31:58.895253 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:31:58.895253 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:31:58.895253 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:31:58.895253 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:31:58.895253 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:31:58.895253 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:31:58.895253 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:31:58.895253 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:31:58.895253 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:31:58.895253 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:31:58.895253 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:31:58.895253 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:31:58.896735 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:31:58.896735 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:31:58.896735 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:31:58.896735 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:31:58.896735 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:31:58.896735 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:31:58.896735 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:31:58.896735 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:31:58.896735 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:31:58.896735 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:31:58.896735 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 02:31:58.896735 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:31:58.896735 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 76/155] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:31:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:31:58.905600 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc3821e-f85e-49b9-9245-1c3d3cbe80f6 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:58.933528 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-379d6035-6484-40d1-a7ad-6b37d25f36ac None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:31:58.936087 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-379d6035-6484-40d1-a7ad-6b37d25f36ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:31:59.066354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:59.087611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:59.103552 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 80/156] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:31:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:31:59.106743 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721a33b-d4a6-446d-83f6-2a5bf4f9ed15 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:59.107069 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721a33b-d4a6-446d-83f6-2a5bf4f9ed15 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:59.107594 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721a33b-d4a6-446d-83f6-2a5bf4f9ed15 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:59.110023 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 81/157] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:31:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:31:59.114268 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc3821e-f85e-49b9-9245-1c3d3cbe80f6 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:59.119439 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f9ba00-a9ca-437d-a282-8e1207848f5e None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:59.119946 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f9ba00-a9ca-437d-a282-8e1207848f5e None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:59.120501 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f9ba00-a9ca-437d-a282-8e1207848f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:59.155195 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-53f9ba00-a9ca-437d-a282-8e1207848f5e None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:31:59.157327 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-53f9ba00-a9ca-437d-a282-8e1207848f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:31:59.158250 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d5a6b-8e2f-4e8a-99db-a72235e1f3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:59.159484 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d5a6b-8e2f-4e8a-99db-a72235e1f3c7 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:59.159947 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d5a6b-8e2f-4e8a-99db-a72235e1f3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:59.160392 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d5a6b-8e2f-4e8a-99db-a72235e1f3c7 None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:59.161287 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d5a6b-8e2f-4e8a-99db-a72235e1f3c7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:31:59.161807 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d5a6b-8e2f-4e8a-99db-a72235e1f3c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:59.163491 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d5a6b-8e2f-4e8a-99db-a72235e1f3c7 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:59.175046 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 77/158] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Sep 22 02:31:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:31:59.189521 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d5a6b-8e2f-4e8a-99db-a72235e1f3c7 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:59.333305 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 82/159] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:31:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 02:31:59.353761 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f9ba00-a9ca-437d-a282-8e1207848f5e None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:59.552050 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d5a6b-8e2f-4e8a-99db-a72235e1f3c7 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:59.662976 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64711678-a3cf-4966-9565-e809d7b8433c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:59.664061 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64711678-a3cf-4966-9565-e809d7b8433c None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:59.664562 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64711678-a3cf-4966-9565-e809d7b8433c None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:59.665049 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64711678-a3cf-4966-9565-e809d7b8433c None admin] PATH_INFO: `/v3/users/c4029608b5a342f78d43dabc3c33197a` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:59.669117 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64711678-a3cf-4966-9565-e809d7b8433c None admin] RBAC: Authorizing `identity:get_user(user_id=c4029608b5a342f78d43dabc3c33197a)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:31:59.671972 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64711678-a3cf-4966-9565-e809d7b8433c None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:31:59.675307 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 78/160] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:31:59 2022] GET /identity/v3/users/c4029608b5a342f78d43dabc3c33197a => generated 289 bytes in 487 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:31:59.685527 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64711678-a3cf-4966-9565-e809d7b8433c None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:59.685821 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f9ba00-a9ca-437d-a282-8e1207848f5e None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:31:59.731791 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:59.740750 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:31:59.796434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199cfe8-3ccd-4a2d-b4f2-3b627341ce24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:31:59.797610 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199cfe8-3ccd-4a2d-b4f2-3b627341ce24 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:31:59.798042 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199cfe8-3ccd-4a2d-b4f2-3b627341ce24 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:31:59.798464 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199cfe8-3ccd-4a2d-b4f2-3b627341ce24 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:31:59.839600 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4199cfe8-3ccd-4a2d-b4f2-3b627341ce24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:31:59.848097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-4199cfe8-3ccd-4a2d-b4f2-3b627341ce24 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 02:31:59.848097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:31:59.848097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:31:59.848097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:31:59.848097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:31:59.848097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:31:59.848097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:31:59.848097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:31:59.848097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:31:59.848097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:31:59.848097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:31:59.848097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:31:59.848097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:31:59.848097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:31:59.848097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:31:59.848097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:31:59.850103 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:31:59.850103 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:31:59.850103 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:31:59.850103 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:31:59.850103 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:31:59.850103 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:31:59.850103 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:31:59.850103 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:31:59.850103 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:31:59.850103 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:31:59.850103 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:31:59.850103 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:31:59.850103 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:31:59.850103 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:31:59.850103 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:31:59.850103 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:31:59.851931 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:31:59.851931 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:31:59.851931 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:31:59.851931 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:31:59.851931 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:31:59.851931 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:31:59.851931 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:31:59.851931 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:31:59.851931 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:31:59.851931 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:31:59.851931 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 02:31:59.851931 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:31:59.851931 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 83/161] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:31:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 496 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:31:59.858517 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199cfe8-3ccd-4a2d-b4f2-3b627341ce24 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:31:59.998993 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64711678-a3cf-4966-9565-e809d7b8433c None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:00.100833 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe246b-1928-450b-90b3-e2c4dbe339e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:00.101408 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe246b-1928-450b-90b3-e2c4dbe339e8 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:00.101698 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe246b-1928-450b-90b3-e2c4dbe339e8 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:00.101981 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe246b-1928-450b-90b3-e2c4dbe339e8 None admin] PATH_INFO: `/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:00.103376 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe246b-1928-450b-90b3-e2c4dbe339e8 None admin] RBAC: Authorizing `identity:get_project(project_id=6a5cfaff6dc7495f9a0648c76c9f6eb7)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:00.104752 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe246b-1928-450b-90b3-e2c4dbe339e8 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:00.109367 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199cfe8-3ccd-4a2d-b4f2-3b627341ce24 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:00.115452 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 79/162] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:31:59 2022] GET /identity/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7 => generated 298 bytes in 432 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:32:00.116400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe246b-1928-450b-90b3-e2c4dbe339e8 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:00.252076 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9190ee-02bf-4c1d-9dde-15d5f9a942ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:00.253142 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9190ee-02bf-4c1d-9dde-15d5f9a942ce None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:00.253604 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9190ee-02bf-4c1d-9dde-15d5f9a942ce None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:00.254061 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9190ee-02bf-4c1d-9dde-15d5f9a942ce None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:00.255516 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9190ee-02bf-4c1d-9dde-15d5f9a942ce None admin] RBAC: Adding query filter params (name=member) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:00.256027 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9190ee-02bf-4c1d-9dde-15d5f9a942ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:00.258664 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9190ee-02bf-4c1d-9dde-15d5f9a942ce None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:00.280745 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:00.302554 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 84/163] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Sep 22 02:31:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 445 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:00.304320 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9190ee-02bf-4c1d-9dde-15d5f9a942ce None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:00.323849 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe246b-1928-450b-90b3-e2c4dbe339e8 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:00.437532 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e177e65-ddb5-4e4d-996d-bc6183a59c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:00.438546 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e177e65-ddb5-4e4d-996d-bc6183a59c08 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:00.439093 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e177e65-ddb5-4e4d-996d-bc6183a59c08 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:00.439596 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e177e65-ddb5-4e4d-996d-bc6183a59c08 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:00.465333 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:00.478556 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e177e65-ddb5-4e4d-996d-bc6183a59c08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:00.498778 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-0e177e65-ddb5-4e4d-996d-bc6183a59c08 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 02:32:00.498778 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:00.498778 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:00.498778 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:00.498778 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:00.498778 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:00.498778 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:00.498778 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:00.498778 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:00.498778 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:00.498778 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:00.498778 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:00.498778 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:00.498778 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:00.498778 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:00.498778 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:00.500687 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:00.500687 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:00.500687 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:00.500687 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:00.500687 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:00.500687 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:00.500687 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:00.500687 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:00.500687 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:00.500687 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:00.500687 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:00.500687 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:00.500687 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:00.500687 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:00.500687 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:00.500687 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:00.502390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:00.502390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:00.502390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:00.502390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:00.502390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:00.502390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:00.502390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:00.502390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:00.502390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:00.502390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:00.502390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 02:32:00.502390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:32:00.502390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 80/164] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:32:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:00.502390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e177e65-ddb5-4e4d-996d-bc6183a59c08 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:00.615458 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9190ee-02bf-4c1d-9dde-15d5f9a942ce None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:00.686582 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e575d-a440-46cd-9e0c-d3c916ce5a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:00.688282 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e575d-a440-46cd-9e0c-d3c916ce5a74 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:00.689363 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e575d-a440-46cd-9e0c-d3c916ce5a74 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:00.690063 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e575d-a440-46cd-9e0c-d3c916ce5a74 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:00.712597 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e575d-a440-46cd-9e0c-d3c916ce5a74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:00.724985 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-4e2e575d-a440-46cd-9e0c-d3c916ce5a74 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 02:32:00.724985 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:00.724985 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:00.724985 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:00.724985 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:00.724985 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:00.724985 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:00.724985 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:00.724985 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:00.724985 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:00.724985 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:00.724985 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:00.724985 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:00.724985 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:00.724985 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:00.724985 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:00.727008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:00.727008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:00.727008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:00.727008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:00.727008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:00.727008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:00.727008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:00.727008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:00.727008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:00.727008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:00.727008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:00.727008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:00.727008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:00.727008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:00.727008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:00.727008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:00.728808 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:00.728808 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:00.728808 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:00.728808 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:00.728808 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:00.728808 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:00.728808 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:00.728808 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:00.728808 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:00.728808 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:00.728808 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 02:32:00.728808 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:32:00.732689 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 85/165] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Sep 22 02:32:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 430 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:00.734512 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e575d-a440-46cd-9e0c-d3c916ce5a74 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:00.850509 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e177e65-ddb5-4e4d-996d-bc6183a59c08 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:00.941776 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad6014-c49b-435e-bb93-ddb0631e7c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:00.942792 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad6014-c49b-435e-bb93-ddb0631e7c36 None admin] REQUEST_METHOD: `PUT` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:00.943302 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad6014-c49b-435e-bb93-ddb0631e7c36 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:00.943760 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad6014-c49b-435e-bb93-ddb0631e7c36 None admin] PATH_INFO: `/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7/users/c4029608b5a342f78d43dabc3c33197a/roles/13d7b9e519274c8bb245c8218571dee8` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:00.951515 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ad6014-c49b-435e-bb93-ddb0631e7c36 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a5cfaff6dc7495f9a0648c76c9f6eb7, user_id=c4029608b5a342f78d43dabc3c33197a, role_id=13d7b9e519274c8bb245c8218571dee8)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:00.954309 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ad6014-c49b-435e-bb93-ddb0631e7c36 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:00.994500 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 81/166] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Sep 22 02:32:00 2022] PUT /identity/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7/users/c4029608b5a342f78d43dabc3c33197a/roles/13d7b9e519274c8bb245c8218571dee8 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 02:32:01.003288 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad6014-c49b-435e-bb93-ddb0631e7c36 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:01.011945 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:01.041619 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:01.056781 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e575d-a440-46cd-9e0c-d3c916ce5a74 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:01.164238 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9009df22-3c80-4112-977d-45b60f46a9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:01.168571 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009df22-3c80-4112-977d-45b60f46a9a9 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:01.169122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009df22-3c80-4112-977d-45b60f46a9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:01.169634 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009df22-3c80-4112-977d-45b60f46a9a9 None admin] PATH_INFO: `/v3/users/c651e80ba10343f4ab505b7e2c378648` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:01.177426 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9009df22-3c80-4112-977d-45b60f46a9a9 None admin] RBAC: Authorizing `identity:get_user(user_id=c651e80ba10343f4ab505b7e2c378648)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:01.182927 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9009df22-3c80-4112-977d-45b60f46a9a9 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:01.192602 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 86/167] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:32:00 2022] GET /identity/v3/users/c651e80ba10343f4ab505b7e2c378648 => generated 275 bytes in 459 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:01.194243 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9009df22-3c80-4112-977d-45b60f46a9a9 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:01.407442 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad6014-c49b-435e-bb93-ddb0631e7c36 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:01.485454 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d6af53-fbaa-4f68-b22e-25ceb988f75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:01.486849 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6af53-fbaa-4f68-b22e-25ceb988f75b None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:01.487377 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6af53-fbaa-4f68-b22e-25ceb988f75b None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:01.487901 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6af53-fbaa-4f68-b22e-25ceb988f75b None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:01.489202 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d6af53-fbaa-4f68-b22e-25ceb988f75b None admin] RBAC: Adding query filter params (name=member) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:01.489755 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d6af53-fbaa-4f68-b22e-25ceb988f75b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:01.493121 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d6af53-fbaa-4f68-b22e-25ceb988f75b None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:01.516896 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 82/168] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Sep 22 02:32:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 522 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:01.519426 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d6af53-fbaa-4f68-b22e-25ceb988f75b None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:01.527643 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9009df22-3c80-4112-977d-45b60f46a9a9 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:01.607939 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ba5d9a-a3fb-4da6-9bfd-c0391deaadb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:01.608652 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba5d9a-a3fb-4da6-9bfd-c0391deaadb8 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:01.609005 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba5d9a-a3fb-4da6-9bfd-c0391deaadb8 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:01.609311 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba5d9a-a3fb-4da6-9bfd-c0391deaadb8 None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:01.610075 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ba5d9a-a3fb-4da6-9bfd-c0391deaadb8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:01.610393 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ba5d9a-a3fb-4da6-9bfd-c0391deaadb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:01.611884 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ba5d9a-a3fb-4da6-9bfd-c0391deaadb8 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:01.632447 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 87/169] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:32:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 439 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:01.633411 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ba5d9a-a3fb-4da6-9bfd-c0391deaadb8 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:01.673305 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:01.746404 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:01.789452 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d6af53-fbaa-4f68-b22e-25ceb988f75b None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:01.900904 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca3aa5e-fc27-4cfc-94bc-54356ea2b67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:01.901551 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca3aa5e-fc27-4cfc-94bc-54356ea2b67c None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:01.901835 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca3aa5e-fc27-4cfc-94bc-54356ea2b67c None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:01.902692 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca3aa5e-fc27-4cfc-94bc-54356ea2b67c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:01.998962 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ba5d9a-a3fb-4da6-9bfd-c0391deaadb8 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:02.108268 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca3aa5e-fc27-4cfc-94bc-54356ea2b67c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:02.109847 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca3aa5e-fc27-4cfc-94bc-54356ea2b67c None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:02.168359 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4052a5-4d4b-4799-81b4-e3086b90d079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:02.169449 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4052a5-4d4b-4799-81b4-e3086b90d079 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:02.169936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4052a5-4d4b-4799-81b4-e3086b90d079 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:02.170416 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4052a5-4d4b-4799-81b4-e3086b90d079 None admin] PATH_INFO: `/v3/users/c651e80ba10343f4ab505b7e2c378648` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:02.174643 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4052a5-4d4b-4799-81b4-e3086b90d079 None admin] RBAC: Authorizing `identity:get_user(user_id=c651e80ba10343f4ab505b7e2c378648)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:02.177678 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4052a5-4d4b-4799-81b4-e3086b90d079 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:02.180803 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 88/170] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:32:01 2022] GET /identity/v3/users/c651e80ba10343f4ab505b7e2c378648 => generated 275 bytes in 548 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:02.188637 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4052a5-4d4b-4799-81b4-e3086b90d079 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:02.242463 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:02.322044 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 83/171] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:32:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 805 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 02:32:02.322968 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca3aa5e-fc27-4cfc-94bc-54356ea2b67c None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:02.568760 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4052a5-4d4b-4799-81b4-e3086b90d079 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:02.583668 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:02.586543 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca3aa5e-fc27-4cfc-94bc-54356ea2b67c None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:02.634662 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6609bd-ab94-4f1f-a371-6d08c4cb586d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:02.635400 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6609bd-ab94-4f1f-a371-6d08c4cb586d None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:02.635687 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6609bd-ab94-4f1f-a371-6d08c4cb586d None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:02.635971 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6609bd-ab94-4f1f-a371-6d08c4cb586d None admin] PATH_INFO: `/v3/users/c651e80ba10343f4ab505b7e2c378648` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:02.637374 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6609bd-ab94-4f1f-a371-6d08c4cb586d None admin] RBAC: Authorizing `identity:get_user(user_id=c651e80ba10343f4ab505b7e2c378648)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:02.639262 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6609bd-ab94-4f1f-a371-6d08c4cb586d None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:02.641972 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 89/172] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:32:02 2022] GET /identity/v3/users/c651e80ba10343f4ab505b7e2c378648 => generated 275 bytes in 454 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:02.642589 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6609bd-ab94-4f1f-a371-6d08c4cb586d None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:02.756094 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd8ea07-020b-4683-bc09-a2ec800ce306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:02.757143 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8ea07-020b-4683-bc09-a2ec800ce306 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:02.757633 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8ea07-020b-4683-bc09-a2ec800ce306 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:02.758109 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8ea07-020b-4683-bc09-a2ec800ce306 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:02.866874 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6609bd-ab94-4f1f-a371-6d08c4cb586d None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:02.875609 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:02.967949 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee425db-84bc-40ef-bd67-0a70c099842f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:02.969021 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee425db-84bc-40ef-bd67-0a70c099842f None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:02.969477 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee425db-84bc-40ef-bd67-0a70c099842f None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:02.969928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee425db-84bc-40ef-bd67-0a70c099842f None admin] PATH_INFO: `/v3/projects/5479a6e9aec54c4c9d3e119308cf2c88` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:02.973166 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee425db-84bc-40ef-bd67-0a70c099842f None admin] RBAC: Authorizing `identity:get_project(project_id=5479a6e9aec54c4c9d3e119308cf2c88)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:02.975902 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee425db-84bc-40ef-bd67-0a70c099842f None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:02.979226 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 90/173] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:32:02 2022] GET /identity/v3/projects/5479a6e9aec54c4c9d3e119308cf2c88 => generated 312 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:02.986454 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee425db-84bc-40ef-bd67-0a70c099842f None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:02.996738 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd8ea07-020b-4683-bc09-a2ec800ce306 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:03.001393 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd8ea07-020b-4683-bc09-a2ec800ce306 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:03.088899 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:03.206916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee425db-84bc-40ef-bd67-0a70c099842f None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:03.231067 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 84/174] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:32:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 908 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 02:32:03.236404 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d6446-6204-4588-9ed0-c15a176ff552 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:03.236812 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d6446-6204-4588-9ed0-c15a176ff552 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:03.237176 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d6446-6204-4588-9ed0-c15a176ff552 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:03.239467 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 85/175] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:03 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:03.240375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3d6446-6204-4588-9ed0-c15a176ff552 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:03.265720 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61df34aa-4693-4d95-87cf-451440dca4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:03.267182 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df34aa-4693-4d95-87cf-451440dca4f1 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:03.267667 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df34aa-4693-4d95-87cf-451440dca4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:03.268160 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df34aa-4693-4d95-87cf-451440dca4f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:03.334552 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:03.366528 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61df34aa-4693-4d95-87cf-451440dca4f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:03.368857 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61df34aa-4693-4d95-87cf-451440dca4f1 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:03.410917 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3d6446-6204-4588-9ed0-c15a176ff552 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:03.418997 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:03.471977 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00e162c-0a2d-4aa2-a0ff-90136c3421cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:03.473413 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e162c-0a2d-4aa2-a0ff-90136c3421cd None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:03.474446 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e162c-0a2d-4aa2-a0ff-90136c3421cd None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:03.475241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e162c-0a2d-4aa2-a0ff-90136c3421cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:03.477603 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00e162c-0a2d-4aa2-a0ff-90136c3421cd None admin] RBAC: Adding query filter params (role.id=83efac3b92eb44e783f4924c9d9ee842, scope.project.id=5479a6e9aec54c4c9d3e119308cf2c88, user.id=c651e80ba10343f4ab505b7e2c378648) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:03.478784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00e162c-0a2d-4aa2-a0ff-90136c3421cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:03.482059 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00e162c-0a2d-4aa2-a0ff-90136c3421cd None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:03.492772 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 86/176] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:32:03 2022] GET /identity/v3/role_assignments?user.id=c651e80ba10343f4ab505b7e2c378648&scope.project.id=5479a6e9aec54c4c9d3e119308cf2c88&role.id=83efac3b92eb44e783f4924c9d9ee842 => generated 608 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:03.495246 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26cf66-31f7-4741-987d-f4ec5caad811 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:03.495848 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26cf66-31f7-4741-987d-f4ec5caad811 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:03.496287 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26cf66-31f7-4741-987d-f4ec5caad811 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:03.497990 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 87/177] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:03.500468 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb8c5c-2147-429e-8641-160612d50e68 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:03.501323 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb8c5c-2147-429e-8641-160612d50e68 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:03.502003 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb8c5c-2147-429e-8641-160612d50e68 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:03.505198 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 88/178] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:03.506876 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb8c5c-2147-429e-8641-160612d50e68 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:03.565555 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 91/179] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:32:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 02:32:03.568876 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce1fe4-186a-4fbb-9a73-168d011c5411 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:03.571333 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce1fe4-186a-4fbb-9a73-168d011c5411 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:03.572057 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce1fe4-186a-4fbb-9a73-168d011c5411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:03.615445 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-f5ce1fe4-186a-4fbb-9a73-168d011c5411 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:03.618473 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-f5ce1fe4-186a-4fbb-9a73-168d011c5411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:03.730397 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb8c5c-2147-429e-8641-160612d50e68 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:03.776929 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 92/180] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:32:03.778579 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299752e9-d3dc-4ca1-b900-0d6e020efb97 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:03.783378 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299752e9-d3dc-4ca1-b900-0d6e020efb97 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:03.783952 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299752e9-d3dc-4ca1-b900-0d6e020efb97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:03.785026 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:03.804393 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5170e31-ee1e-4942-b35f-96bea29cf3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:03.805549 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5170e31-ee1e-4942-b35f-96bea29cf3cc None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:03.806056 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5170e31-ee1e-4942-b35f-96bea29cf3cc None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:03.806563 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5170e31-ee1e-4942-b35f-96bea29cf3cc None admin] PATH_INFO: `/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:03.809792 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5170e31-ee1e-4942-b35f-96bea29cf3cc None admin] RBAC: Authorizing `identity:get_project(project_id=6a5cfaff6dc7495f9a0648c76c9f6eb7)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:03.812490 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5170e31-ee1e-4942-b35f-96bea29cf3cc None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:03.813643 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-299752e9-d3dc-4ca1-b900-0d6e020efb97 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:03.814922 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-299752e9-d3dc-4ca1-b900-0d6e020efb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:03.817123 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 89/181] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:32:03 2022] GET /identity/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7 => generated 298 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:03.819747 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e28444-a595-4324-892b-5c48aa6ba583 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:03.820286 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e28444-a595-4324-892b-5c48aa6ba583 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:03.820769 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e28444-a595-4324-892b-5c48aa6ba583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:03.846802 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-44e28444-a595-4324-892b-5c48aa6ba583 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:03.848066 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-44e28444-a595-4324-892b-5c48aa6ba583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:03.956078 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:03.972390 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 93/182] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:32:03.974475 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299752e9-d3dc-4ca1-b900-0d6e020efb97 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:03.978985 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 90/183] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:32:03.988915 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e28444-a595-4324-892b-5c48aa6ba583 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:04.256234 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e28444-a595-4324-892b-5c48aa6ba583 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:04.270811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299752e9-d3dc-4ca1-b900-0d6e020efb97 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:04.343409 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:04.353918 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894b2da-1c29-4326-9773-f7c004012245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:04.354589 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894b2da-1c29-4326-9773-f7c004012245 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:04.354939 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894b2da-1c29-4326-9773-f7c004012245 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:04.355230 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894b2da-1c29-4326-9773-f7c004012245 None admin] PATH_INFO: `/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:04.356495 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e894b2da-1c29-4326-9773-f7c004012245 None admin] RBAC: Authorizing `identity:get_project(project_id=6a5cfaff6dc7495f9a0648c76c9f6eb7)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:04.357952 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e894b2da-1c29-4326-9773-f7c004012245 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:04.359719 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 94/184] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:32:03 2022] GET /identity/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7 => generated 298 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:04.360622 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894b2da-1c29-4326-9773-f7c004012245 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:04.387945 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dedf8f-80f0-4e8e-ac65-190ca5f25fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:04.389355 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dedf8f-80f0-4e8e-ac65-190ca5f25fd0 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:04.390034 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dedf8f-80f0-4e8e-ac65-190ca5f25fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:04.390690 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dedf8f-80f0-4e8e-ac65-190ca5f25fd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:04.392268 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dedf8f-80f0-4e8e-ac65-190ca5f25fd0 None admin] RBAC: Adding query filter params (role.id=83efac3b92eb44e783f4924c9d9ee842, scope.project.id=6a5cfaff6dc7495f9a0648c76c9f6eb7, user.id=c651e80ba10343f4ab505b7e2c378648) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:04.392933 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dedf8f-80f0-4e8e-ac65-190ca5f25fd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:04.395452 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dedf8f-80f0-4e8e-ac65-190ca5f25fd0 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:04.419628 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 91/185] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:32:03 2022] GET /identity/v3/role_assignments?user.id=c651e80ba10343f4ab505b7e2c378648&scope.project.id=6a5cfaff6dc7495f9a0648c76c9f6eb7&role.id=83efac3b92eb44e783f4924c9d9ee842 => generated 608 bytes in 432 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:04.421313 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dedf8f-80f0-4e8e-ac65-190ca5f25fd0 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:04.439457 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:04.488300 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894b2da-1c29-4326-9773-f7c004012245 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:04.537810 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd40dd-73d8-4eb5-9594-a8e11132fdc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:04.538393 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd40dd-73d8-4eb5-9594-a8e11132fdc0 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:04.538677 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd40dd-73d8-4eb5-9594-a8e11132fdc0 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:04.538986 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd40dd-73d8-4eb5-9594-a8e11132fdc0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:04.562876 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dd40dd-73d8-4eb5-9594-a8e11132fdc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:04.568189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-f9dd40dd-73d8-4eb5-9594-a8e11132fdc0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 02:32:04.568189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:04.568189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:04.568189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:04.568189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:04.568189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:04.568189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:04.568189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:04.568189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:04.568189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:04.568189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:04.568189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:04.568189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:04.568189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:04.568189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:04.568189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:04.570319 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:04.570319 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:04.570319 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:04.570319 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:04.570319 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:04.570319 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:04.570319 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:04.570319 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:04.570319 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:04.570319 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:04.570319 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:04.570319 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:04.570319 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:04.570319 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:04.570319 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:04.570319 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:04.571724 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:04.571724 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:04.571724 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:04.571724 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:04.571724 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:04.571724 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:04.571724 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:04.571724 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:04.571724 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:04.571724 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:04.571724 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 02:32:04.571724 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:32:04.575422 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 95/186] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Sep 22 02:32:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:04.576742 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd40dd-73d8-4eb5-9594-a8e11132fdc0 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:04.690126 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dedf8f-80f0-4e8e-ac65-190ca5f25fd0 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:04.744920 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971f5cc-6e93-4c85-8d96-318002a3c752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:04.746063 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd40dd-73d8-4eb5-9594-a8e11132fdc0 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:04.746176 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971f5cc-6e93-4c85-8d96-318002a3c752 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:04.746469 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971f5cc-6e93-4c85-8d96-318002a3c752 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:04.746621 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971f5cc-6e93-4c85-8d96-318002a3c752 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:04.747497 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9971f5cc-6e93-4c85-8d96-318002a3c752 None admin] RBAC: Adding query filter params (role.id=adec3004b56243fa923bd9f0082a2728, scope.project.id=6a5cfaff6dc7495f9a0648c76c9f6eb7, user.id=c651e80ba10343f4ab505b7e2c378648) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:04.747876 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9971f5cc-6e93-4c85-8d96-318002a3c752 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:04.749670 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9971f5cc-6e93-4c85-8d96-318002a3c752 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:04.762738 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 92/187] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:32:04 2022] GET /identity/v3/role_assignments?user.id=c651e80ba10343f4ab505b7e2c378648&scope.project.id=6a5cfaff6dc7495f9a0648c76c9f6eb7&role.id=adec3004b56243fa923bd9f0082a2728 => generated 608 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:04.763960 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971f5cc-6e93-4c85-8d96-318002a3c752 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:04.795345 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:04.813886 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e7cf9-e4d7-4540-8b81-119519c936f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:04.815138 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e7cf9-e4d7-4540-8b81-119519c936f8 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:04.815606 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e7cf9-e4d7-4540-8b81-119519c936f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:04.816024 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e7cf9-e4d7-4540-8b81-119519c936f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:04.832750 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6e7cf9-e4d7-4540-8b81-119519c936f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:04.839102 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-ca6e7cf9-e4d7-4540-8b81-119519c936f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:32:04.839102 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:04.839102 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:04.839102 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:04.839102 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:04.839102 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:04.839102 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:04.839102 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:04.839102 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:04.839102 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:04.839102 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:04.839102 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:04.839102 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:04.839102 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:04.839102 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:04.839102 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:04.844373 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:04.844373 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:04.844373 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:04.844373 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:04.844373 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:04.844373 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:04.844373 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:04.844373 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:04.844373 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:04.844373 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:04.844373 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:04.844373 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:04.844373 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:04.844373 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:04.844373 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:04.844373 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:04.845172 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:04.845172 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:04.845172 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:04.845172 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:04.845172 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:04.845172 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:04.845172 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:04.845172 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:04.845172 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:04.845172 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:04.845172 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:32:04.845172 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:32:04.849201 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 96/188] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Sep 22 02:32:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:04.850298 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e7cf9-e4d7-4540-8b81-119519c936f8 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:04.868940 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:04.953410 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971f5cc-6e93-4c85-8d96-318002a3c752 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:05.033266 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4261e4-0c0e-45f1-a891-fa83898c27b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:05.034388 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4261e4-0c0e-45f1-a891-fa83898c27b5 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:05.034861 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4261e4-0c0e-45f1-a891-fa83898c27b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:05.035283 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4261e4-0c0e-45f1-a891-fa83898c27b5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:05.039093 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4261e4-0c0e-45f1-a891-fa83898c27b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:05.041681 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4261e4-0c0e-45f1-a891-fa83898c27b5 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:05.045554 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 93/189] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:32:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:05.051116 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4261e4-0c0e-45f1-a891-fa83898c27b5 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:05.063652 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e7cf9-e4d7-4540-8b81-119519c936f8 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:05.112427 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f67657-d8dc-4d59-9b13-fcbc1f58ffc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:05.113100 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f67657-d8dc-4d59-9b13-fcbc1f58ffc4 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:05.113381 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f67657-d8dc-4d59-9b13-fcbc1f58ffc4 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:05.113654 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f67657-d8dc-4d59-9b13-fcbc1f58ffc4 None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:05.114352 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f67657-d8dc-4d59-9b13-fcbc1f58ffc4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:05.114668 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f67657-d8dc-4d59-9b13-fcbc1f58ffc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:05.116146 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f67657-d8dc-4d59-9b13-fcbc1f58ffc4 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:05.132233 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 97/190] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:32:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:05.136641 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f67657-d8dc-4d59-9b13-fcbc1f58ffc4 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:05.230537 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:05.275792 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4261e4-0c0e-45f1-a891-fa83898c27b5 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:05.303657 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:05.321880 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b95fd-5aa9-4fee-a935-f1c700b8af36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:05.322751 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b95fd-5aa9-4fee-a935-f1c700b8af36 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:05.323188 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b95fd-5aa9-4fee-a935-f1c700b8af36 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:05.323510 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b95fd-5aa9-4fee-a935-f1c700b8af36 None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:05.324463 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9b95fd-5aa9-4fee-a935-f1c700b8af36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:05.324909 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9b95fd-5aa9-4fee-a935-f1c700b8af36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:05.326693 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9b95fd-5aa9-4fee-a935-f1c700b8af36 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:05.328147 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f67657-d8dc-4d59-9b13-fcbc1f58ffc4 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:05.338551 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 94/191] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:32:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:05.342147 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b95fd-5aa9-4fee-a935-f1c700b8af36 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:05.382208 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616c248-ffac-454a-b852-3917c6beb0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:05.382870 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616c248-ffac-454a-b852-3917c6beb0da None admin] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:05.383166 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616c248-ffac-454a-b852-3917c6beb0da None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:05.383450 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616c248-ffac-454a-b852-3917c6beb0da None admin] PATH_INFO: `/v3/users` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:05.384130 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d616c248-ffac-454a-b852-3917c6beb0da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:05.385892 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d616c248-ffac-454a-b852-3917c6beb0da None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:05.421530 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 98/192] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Sep 22 02:32:05 2022] POST /identity/v3/users => generated 297 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:32:05.421530 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616c248-ffac-454a-b852-3917c6beb0da None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:05.497044 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b95fd-5aa9-4fee-a935-f1c700b8af36 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:05.568302 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43102a8a-b51a-410e-8170-0d084a78c185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:05.569349 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43102a8a-b51a-410e-8170-0d084a78c185 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:05.569807 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43102a8a-b51a-410e-8170-0d084a78c185 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:05.571264 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43102a8a-b51a-410e-8170-0d084a78c185 None admin] PATH_INFO: `/v3/users/142866c87db6452c8fff5c8d3c4aa04e` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:05.594329 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:05.596292 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43102a8a-b51a-410e-8170-0d084a78c185 None admin] RBAC: Authorizing `identity:get_user(user_id=142866c87db6452c8fff5c8d3c4aa04e)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:05.598911 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43102a8a-b51a-410e-8170-0d084a78c185 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:05.600293 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616c248-ffac-454a-b852-3917c6beb0da None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:05.601883 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 95/193] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:32:05 2022] GET /identity/v3/users/142866c87db6452c8fff5c8d3c4aa04e => generated 283 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:05.609556 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41a70c-cc03-4572-804e-4ba4382ce90a None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:05.610074 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41a70c-cc03-4572-804e-4ba4382ce90a None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:05.610440 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41a70c-cc03-4572-804e-4ba4382ce90a None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:05.618700 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 96/194] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:05 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:05.618700 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41a70c-cc03-4572-804e-4ba4382ce90a None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:05.668919 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453372bf-0620-461b-9db7-7ab40cdc1427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:05.671147 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453372bf-0620-461b-9db7-7ab40cdc1427 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:05.671147 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453372bf-0620-461b-9db7-7ab40cdc1427 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:05.671147 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453372bf-0620-461b-9db7-7ab40cdc1427 None admin] PATH_INFO: `/v3/users/142866c87db6452c8fff5c8d3c4aa04e` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:05.672121 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453372bf-0620-461b-9db7-7ab40cdc1427 None admin] RBAC: Authorizing `identity:get_user(user_id=142866c87db6452c8fff5c8d3c4aa04e)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:05.674711 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453372bf-0620-461b-9db7-7ab40cdc1427 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:05.677862 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 99/195] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:32:05 2022] GET /identity/v3/users/142866c87db6452c8fff5c8d3c4aa04e => generated 283 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:05.680303 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750bd049-defa-4d04-ae13-89453aadafcc None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:05.680938 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750bd049-defa-4d04-ae13-89453aadafcc None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:05.681424 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750bd049-defa-4d04-ae13-89453aadafcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:05.706439 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:05.712371 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-750bd049-defa-4d04-ae13-89453aadafcc None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:05.714107 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-750bd049-defa-4d04-ae13-89453aadafcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:05.801773 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41a70c-cc03-4572-804e-4ba4382ce90a None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:05.844578 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 100/196] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:32:05.846306 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750bd049-defa-4d04-ae13-89453aadafcc None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:05.877461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43f3a8-d518-46a8-87c4-c5bcffc73eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:05.878120 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43f3a8-d518-46a8-87c4-c5bcffc73eb3 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:05.878407 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43f3a8-d518-46a8-87c4-c5bcffc73eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:05.878705 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43f3a8-d518-46a8-87c4-c5bcffc73eb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:06.082206 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a43f3a8-d518-46a8-87c4-c5bcffc73eb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:06.085138 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a43f3a8-d518-46a8-87c4-c5bcffc73eb3 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:06.141426 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:06.149412 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750bd049-defa-4d04-ae13-89453aadafcc None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:06.202790 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:06.219455 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e7558-eee4-4b4d-b9ed-2e61b92cd5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:06.220094 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e7558-eee4-4b4d-b9ed-2e61b92cd5f4 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:06.220377 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e7558-eee4-4b4d-b9ed-2e61b92cd5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:06.220655 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e7558-eee4-4b4d-b9ed-2e61b92cd5f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:06.298981 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 97/197] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:32:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 681 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 02:32:06.300032 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43f3a8-d518-46a8-87c4-c5bcffc73eb3 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:06.323018 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e7558-eee4-4b4d-b9ed-2e61b92cd5f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:06.325511 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e7558-eee4-4b4d-b9ed-2e61b92cd5f4 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:06.422862 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43f3a8-d518-46a8-87c4-c5bcffc73eb3 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:06.450388 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:06.455621 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 101/198] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:32:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 02:32:06.456546 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e7558-eee4-4b4d-b9ed-2e61b92cd5f4 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:06.477280 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44575249-d1c7-4baf-a103-ece3d348255b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:06.478275 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44575249-d1c7-4baf-a103-ece3d348255b None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:06.478561 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44575249-d1c7-4baf-a103-ece3d348255b None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:06.479041 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44575249-d1c7-4baf-a103-ece3d348255b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:06.494458 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:06.500878 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44575249-d1c7-4baf-a103-ece3d348255b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:06.507328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-44575249-d1c7-4baf-a103-ece3d348255b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 02:32:06.507328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:06.507328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:06.507328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:06.507328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:06.507328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:06.507328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:06.507328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:06.507328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:06.507328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:06.507328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:06.507328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:06.507328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:06.507328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:06.507328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:06.507328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:06.508883 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:06.508883 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:06.508883 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:06.508883 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:06.508883 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:06.508883 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:06.508883 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:06.508883 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:06.508883 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:06.508883 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:06.508883 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:06.508883 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:06.508883 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:06.508883 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:06.508883 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:06.508883 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:06.510279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:06.510279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:06.510279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:06.510279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:06.510279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:06.510279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:06.510279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:06.510279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:06.510279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:06.510279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:06.510279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 02:32:06.510279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:32:06.513006 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 98/199] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:32:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:06.513006 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44575249-d1c7-4baf-a103-ece3d348255b None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:06.583691 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e7558-eee4-4b4d-b9ed-2e61b92cd5f4 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:06.635623 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d35c2-1df0-47a1-877e-923f29addc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:06.637291 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d35c2-1df0-47a1-877e-923f29addc48 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:06.637291 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d35c2-1df0-47a1-877e-923f29addc48 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:06.637832 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d35c2-1df0-47a1-877e-923f29addc48 None admin] PATH_INFO: `/v3/projects/d861cd3fc0a047d6bf06a6250256ee77` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:06.642878 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d35c2-1df0-47a1-877e-923f29addc48 None admin] RBAC: Authorizing `identity:get_project(project_id=d861cd3fc0a047d6bf06a6250256ee77)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:06.642878 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d35c2-1df0-47a1-877e-923f29addc48 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:06.643734 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 102/200] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:32:06 2022] GET /identity/v3/projects/d861cd3fc0a047d6bf06a6250256ee77 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:06.645107 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d35c2-1df0-47a1-877e-923f29addc48 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:06.656688 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44575249-d1c7-4baf-a103-ece3d348255b None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:06.696568 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be29ed20-9222-4133-8d62-9354a4544a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:06.697146 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29ed20-9222-4133-8d62-9354a4544a5d None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:06.697411 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29ed20-9222-4133-8d62-9354a4544a5d None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:06.697698 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29ed20-9222-4133-8d62-9354a4544a5d None admin] PATH_INFO: `/v3/projects/d861cd3fc0a047d6bf06a6250256ee77` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:06.699616 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be29ed20-9222-4133-8d62-9354a4544a5d None admin] RBAC: Authorizing `identity:get_project(project_id=d861cd3fc0a047d6bf06a6250256ee77)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:06.701141 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be29ed20-9222-4133-8d62-9354a4544a5d None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:06.702962 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 99/201] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:32:06 2022] GET /identity/v3/projects/d861cd3fc0a047d6bf06a6250256ee77 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:06.703980 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be29ed20-9222-4133-8d62-9354a4544a5d None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:06.796246 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:06.798179 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:06.841071 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be29ed20-9222-4133-8d62-9354a4544a5d None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:06.855959 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d35c2-1df0-47a1-877e-923f29addc48 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:06.883062 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec9edb-650d-4fbe-b273-91a52dd2c8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:06.884056 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec9edb-650d-4fbe-b273-91a52dd2c8de None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:06.884521 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec9edb-650d-4fbe-b273-91a52dd2c8de None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:06.884972 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec9edb-650d-4fbe-b273-91a52dd2c8de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:06.885921 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec9edb-650d-4fbe-b273-91a52dd2c8de None admin] RBAC: Adding query filter params (role.id=adec3004b56243fa923bd9f0082a2728, scope.project.id=d861cd3fc0a047d6bf06a6250256ee77, user.id=142866c87db6452c8fff5c8d3c4aa04e) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:06.886503 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec9edb-650d-4fbe-b273-91a52dd2c8de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:06.888259 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec9edb-650d-4fbe-b273-91a52dd2c8de None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:06.895220 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 100/202] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:32:06 2022] GET /identity/v3/role_assignments?user.id=142866c87db6452c8fff5c8d3c4aa04e&scope.project.id=d861cd3fc0a047d6bf06a6250256ee77&role.id=adec3004b56243fa923bd9f0082a2728 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:06.896489 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec9edb-650d-4fbe-b273-91a52dd2c8de None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:06.941174 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56894776-7b51-4203-9360-7b819ca2e7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:06.942279 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56894776-7b51-4203-9360-7b819ca2e7a9 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:06.942784 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56894776-7b51-4203-9360-7b819ca2e7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:06.943356 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56894776-7b51-4203-9360-7b819ca2e7a9 None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:06.944526 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56894776-7b51-4203-9360-7b819ca2e7a9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:06.945123 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56894776-7b51-4203-9360-7b819ca2e7a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:06.947884 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56894776-7b51-4203-9360-7b819ca2e7a9 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:06.965981 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 103/203] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Sep 22 02:32:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:06.975246 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56894776-7b51-4203-9360-7b819ca2e7a9 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:07.034714 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec9edb-650d-4fbe-b273-91a52dd2c8de None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:07.072194 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:07.079242 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a07e3-149a-4693-91e1-1301526a0ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:07.080237 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a07e3-149a-4693-91e1-1301526a0ce0 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:07.080711 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a07e3-149a-4693-91e1-1301526a0ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:07.081834 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a07e3-149a-4693-91e1-1301526a0ce0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:07.082857 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164a07e3-149a-4693-91e1-1301526a0ce0 None admin] RBAC: Adding query filter params (role.id=31a687548f55493b8ea666fe7e350303, scope.project.id=d861cd3fc0a047d6bf06a6250256ee77, user.id=142866c87db6452c8fff5c8d3c4aa04e) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:07.083674 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164a07e3-149a-4693-91e1-1301526a0ce0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:07.085567 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164a07e3-149a-4693-91e1-1301526a0ce0 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:07.096524 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 101/204] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:32:06 2022] GET /identity/v3/role_assignments?user.id=142866c87db6452c8fff5c8d3c4aa04e&scope.project.id=d861cd3fc0a047d6bf06a6250256ee77&role.id=31a687548f55493b8ea666fe7e350303 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:07.147724 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56894776-7b51-4203-9360-7b819ca2e7a9 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:07.180303 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:07.195361 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bcba90-87c3-4929-a695-0c0c7feb09ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:07.196778 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bcba90-87c3-4929-a695-0c0c7feb09ac None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:07.197377 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bcba90-87c3-4929-a695-0c0c7feb09ac None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:07.198147 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bcba90-87c3-4929-a695-0c0c7feb09ac None admin] PATH_INFO: `/v3/users/c4029608b5a342f78d43dabc3c33197a` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:07.200696 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bcba90-87c3-4929-a695-0c0c7feb09ac None admin] RBAC: Authorizing `identity:get_user(user_id=c4029608b5a342f78d43dabc3c33197a)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:07.202202 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bcba90-87c3-4929-a695-0c0c7feb09ac None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:07.204193 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 104/205] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:32:06 2022] GET /identity/v3/users/c4029608b5a342f78d43dabc3c33197a => generated 289 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:07.209928 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a07e3-149a-4693-91e1-1301526a0ce0 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:07.351200 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a07e3-149a-4693-91e1-1301526a0ce0 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:07.398845 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2351d-c459-4a09-a5f0-cacda10fe759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:07.400174 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2351d-c459-4a09-a5f0-cacda10fe759 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:07.400661 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2351d-c459-4a09-a5f0-cacda10fe759 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:07.401132 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2351d-c459-4a09-a5f0-cacda10fe759 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:07.491629 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:07.500232 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec2351d-c459-4a09-a5f0-cacda10fe759 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:07.502135 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec2351d-c459-4a09-a5f0-cacda10fe759 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:07.618143 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 102/206] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:32:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 22 02:32:07.624713 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bcba90-87c3-4929-a695-0c0c7feb09ac None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:07.785970 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bcba90-87c3-4929-a695-0c0c7feb09ac None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:07.834586 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4483e5-7e8e-4d08-8505-31fcc717e73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:07.835281 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4483e5-7e8e-4d08-8505-31fcc717e73b None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:07.835569 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4483e5-7e8e-4d08-8505-31fcc717e73b None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:07.835851 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4483e5-7e8e-4d08-8505-31fcc717e73b None admin] PATH_INFO: `/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:07.837057 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4483e5-7e8e-4d08-8505-31fcc717e73b None admin] RBAC: Authorizing `identity:get_project(project_id=6a5cfaff6dc7495f9a0648c76c9f6eb7)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:07.838558 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4483e5-7e8e-4d08-8505-31fcc717e73b None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:07.841135 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 105/207] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:32:07 2022] GET /identity/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:07.847151 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2351d-c459-4a09-a5f0-cacda10fe759 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:07.961569 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2351d-c459-4a09-a5f0-cacda10fe759 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:07.971864 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:08.015521 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a36fe-9100-4310-849f-acd077d4bdf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:08.016156 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a36fe-9100-4310-849f-acd077d4bdf1 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:08.016440 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a36fe-9100-4310-849f-acd077d4bdf1 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:08.016714 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a36fe-9100-4310-849f-acd077d4bdf1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:08.017460 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a36fe-9100-4310-849f-acd077d4bdf1 None admin] RBAC: Adding query filter params (role.id=13d7b9e519274c8bb245c8218571dee8, scope.project.id=6a5cfaff6dc7495f9a0648c76c9f6eb7, user.id=c4029608b5a342f78d43dabc3c33197a) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:08.017770 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a36fe-9100-4310-849f-acd077d4bdf1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:08.019289 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a36fe-9100-4310-849f-acd077d4bdf1 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:08.029767 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 103/208] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:32:07 2022] GET /identity/v3/role_assignments?user.id=c4029608b5a342f78d43dabc3c33197a&scope.project.id=6a5cfaff6dc7495f9a0648c76c9f6eb7&role.id=13d7b9e519274c8bb245c8218571dee8 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:32:09.126473 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9088e3fd-6d2e-4a05-a888-b92631c5d250 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:09.128018 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9088e3fd-6d2e-4a05-a888-b92631c5d250 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:09.128798 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9088e3fd-6d2e-4a05-a888-b92631c5d250 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:09.131482 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 106/209] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:09.141617 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf2a65-8df8-4665-91b8-336e09211fc6 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:09.143112 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf2a65-8df8-4665-91b8-336e09211fc6 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:09.143646 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf2a65-8df8-4665-91b8-336e09211fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:09.170316 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4597080-e09d-4e29-aa4d-a1c9052c1bc6 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:09.171144 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4597080-e09d-4e29-aa4d-a1c9052c1bc6 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:09.171144 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4597080-e09d-4e29-aa4d-a1c9052c1bc6 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:09.172988 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 107/210] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:09.180619 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a00c6-7faa-4654-9415-fffc5e093f8d None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:09.181069 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a00c6-7faa-4654-9415-fffc5e093f8d None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:09.181534 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a00c6-7faa-4654-9415-fffc5e093f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:09.184862 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-5faf2a65-8df8-4665-91b8-336e09211fc6 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:09.187336 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-5faf2a65-8df8-4665-91b8-336e09211fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:09.212986 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-3c8a00c6-7faa-4654-9415-fffc5e093f8d None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:09.215652 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-3c8a00c6-7faa-4654-9415-fffc5e093f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:09.290445 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 104/211] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:32:09.343533 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:09.354207 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 108/212] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:32:09.810873 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faf2a65-8df8-4665-91b8-336e09211fc6 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:09.859745 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a00c6-7faa-4654-9415-fffc5e093f8d None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:09.940621 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:09.955113 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faf2a65-8df8-4665-91b8-336e09211fc6 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:10.021448 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01ad9a-8e98-48d4-a8a7-6981566cbb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:10.022524 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01ad9a-8e98-48d4-a8a7-6981566cbb11 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:10.023040 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01ad9a-8e98-48d4-a8a7-6981566cbb11 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:10.023503 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01ad9a-8e98-48d4-a8a7-6981566cbb11 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:10.034390 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a00c6-7faa-4654-9415-fffc5e093f8d None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:10.043791 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea01ad9a-8e98-48d4-a8a7-6981566cbb11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:10.054000 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-ea01ad9a-8e98-48d4-a8a7-6981566cbb11 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 02:32:10.054000 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:10.054000 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:10.054000 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:10.054000 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:10.054000 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:10.054000 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:10.054000 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:10.054000 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:10.054000 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:10.054000 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:10.054000 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:10.054000 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:10.054000 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:10.054000 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:10.054000 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:10.055907 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:10.055907 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:10.055907 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:10.055907 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:10.055907 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:10.055907 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:10.055907 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:10.055907 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:10.055907 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:10.055907 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:10.055907 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:10.055907 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:10.055907 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:10.055907 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:10.055907 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:10.055907 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:10.057481 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:10.057481 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:10.057481 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:10.057481 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:10.057481 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:10.057481 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:10.057481 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:10.057481 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:10.057481 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:10.057481 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:10.057481 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 02:32:10.057481 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:32:10.062101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 105/213] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:32:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:10.070980 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01ad9a-8e98-48d4-a8a7-6981566cbb11 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:10.076248 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565d5b52-d8bd-4956-a875-b4eecfb8258f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:10.077215 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d5b52-d8bd-4956-a875-b4eecfb8258f None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:10.077643 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d5b52-d8bd-4956-a875-b4eecfb8258f None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:10.077977 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d5b52-d8bd-4956-a875-b4eecfb8258f None admin] PATH_INFO: `/v3/domains/default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:10.079669 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565d5b52-d8bd-4956-a875-b4eecfb8258f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:10.081730 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565d5b52-d8bd-4956-a875-b4eecfb8258f None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:10.084497 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 109/214] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:32:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:32:10.091709 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565d5b52-d8bd-4956-a875-b4eecfb8258f None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:10.187406 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:10.205324 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01ad9a-8e98-48d4-a8a7-6981566cbb11 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:10.255152 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef7d48-cb18-4d72-b94d-b2ee362bc69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:10.256131 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef7d48-cb18-4d72-b94d-b2ee362bc69d None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:10.256566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef7d48-cb18-4d72-b94d-b2ee362bc69d None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:10.256988 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef7d48-cb18-4d72-b94d-b2ee362bc69d None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:10.258103 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef7d48-cb18-4d72-b94d-b2ee362bc69d None admin] RBAC: Adding query filter params (name=member) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:10.258580 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef7d48-cb18-4d72-b94d-b2ee362bc69d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:10.259646 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565d5b52-d8bd-4956-a875-b4eecfb8258f None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:10.261511 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef7d48-cb18-4d72-b94d-b2ee362bc69d None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:10.281343 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 106/215] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Sep 22 02:32:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:10.281343 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de10558-8ad5-44cf-a5e8-21da31506c5d None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:10.281343 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de10558-8ad5-44cf-a5e8-21da31506c5d None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:10.281343 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de10558-8ad5-44cf-a5e8-21da31506c5d None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:10.282106 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 107/216] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:10.284483 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de10558-8ad5-44cf-a5e8-21da31506c5d None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:10.303547 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:10.340898 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725910b-d280-47fc-a365-7b18bdb61067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:10.341417 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725910b-d280-47fc-a365-7b18bdb61067 None admin] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:10.341887 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725910b-d280-47fc-a365-7b18bdb61067 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:10.342382 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725910b-d280-47fc-a365-7b18bdb61067 None admin] PATH_INFO: `/v3/users` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:10.343698 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7725910b-d280-47fc-a365-7b18bdb61067 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:10.346471 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7725910b-d280-47fc-a365-7b18bdb61067 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:10.387280 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 110/217] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Sep 22 02:32:10 2022] POST /identity/v3/users => generated 297 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 02:32:10.389525 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085d958-65f6-4fb5-ae03-1c6d03feedfc None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:10.389955 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085d958-65f6-4fb5-ae03-1c6d03feedfc None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:10.390445 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085d958-65f6-4fb5-ae03-1c6d03feedfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:10.426775 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-6085d958-65f6-4fb5-ae03-1c6d03feedfc None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:10.429099 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-6085d958-65f6-4fb5-ae03-1c6d03feedfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:10.458280 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de10558-8ad5-44cf-a5e8-21da31506c5d None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:10.531869 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0491f660-bd4a-4b61-9295-89d33b624d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:10.532540 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491f660-bd4a-4b61-9295-89d33b624d86 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:10.532827 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491f660-bd4a-4b61-9295-89d33b624d86 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:10.533107 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491f660-bd4a-4b61-9295-89d33b624d86 None admin] PATH_INFO: `/v3/users/a97426e846c0416387b714de740cd043` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:10.551472 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0491f660-bd4a-4b61-9295-89d33b624d86 None admin] RBAC: Authorizing `identity:get_user(user_id=a97426e846c0416387b714de740cd043)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:10.553389 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0491f660-bd4a-4b61-9295-89d33b624d86 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:10.555401 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 108/218] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:32:10 2022] GET /identity/v3/users/a97426e846c0416387b714de740cd043 => generated 297 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:10.570509 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0491f660-bd4a-4b61-9295-89d33b624d86 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:10.598509 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 111/219] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:32:10.690064 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:10.731242 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0491f660-bd4a-4b61-9295-89d33b624d86 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:10.772577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd958450-8052-4514-b32e-8225e476d341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:10.773283 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd958450-8052-4514-b32e-8225e476d341 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:10.773568 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd958450-8052-4514-b32e-8225e476d341 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:10.773847 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd958450-8052-4514-b32e-8225e476d341 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:10.860539 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd958450-8052-4514-b32e-8225e476d341 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:10.861993 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd958450-8052-4514-b32e-8225e476d341 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:10.874374 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2a653-b3ad-4288-b3d0-ed5aaa592b53 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:10.874894 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2a653-b3ad-4288-b3d0-ed5aaa592b53 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:10.875367 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2a653-b3ad-4288-b3d0-ed5aaa592b53 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:10.877872 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 112/220] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:10.887053 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6cd9ad-2533-4fee-b0f5-83981f5b6204 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:10.887327 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6cd9ad-2533-4fee-b0f5-83981f5b6204 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:10.887624 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6cd9ad-2533-4fee-b0f5-83981f5b6204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:10.920700 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-dd6cd9ad-2533-4fee-b0f5-83981f5b6204 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:10.922054 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-dd6cd9ad-2533-4fee-b0f5-83981f5b6204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:10.962045 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:10.978425 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:11.009652 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 109/221] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:32:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 22 02:32:11.010571 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd958450-8052-4514-b32e-8225e476d341 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:11.034647 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 113/222] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 02:32:11.040704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6cd9ad-2533-4fee-b0f5-83981f5b6204 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:11.155528 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd958450-8052-4514-b32e-8225e476d341 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:11.214763 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55797d8-e76a-49c4-9233-de33b8b44434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:11.216694 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55797d8-e76a-49c4-9233-de33b8b44434 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:11.216978 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55797d8-e76a-49c4-9233-de33b8b44434 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:11.217255 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55797d8-e76a-49c4-9233-de33b8b44434 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:11.248638 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55797d8-e76a-49c4-9233-de33b8b44434 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:11.260667 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-c55797d8-e76a-49c4-9233-de33b8b44434 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:32:11.260667 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:11.260667 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:11.260667 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:11.260667 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:11.260667 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:11.260667 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:11.260667 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:11.260667 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:11.260667 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:11.260667 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:11.260667 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:11.260667 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:11.260667 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:11.260667 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:11.260667 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:11.261985 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:11.261985 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:11.261985 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:11.261985 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:11.261985 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:11.261985 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:11.261985 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:11.261985 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:11.261985 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:11.261985 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:11.261985 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:11.261985 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:11.261985 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:11.261985 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:11.261985 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:11.261985 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:11.262803 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:11.262803 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:11.262803 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:11.262803 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:11.262803 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:11.262803 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:11.262803 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:11.262803 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:11.262803 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:11.262803 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:11.262803 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:32:11.262803 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:32:11.262803 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 110/223] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Sep 22 02:32:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:11.272395 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55797d8-e76a-49c4-9233-de33b8b44434 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:11.319379 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6cd9ad-2533-4fee-b0f5-83981f5b6204 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:11.353581 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:11.392891 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e633d-318d-495c-9990-3ed99c03cde0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:11.393988 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e633d-318d-495c-9990-3ed99c03cde0 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:11.394483 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e633d-318d-495c-9990-3ed99c03cde0 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:11.395014 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e633d-318d-495c-9990-3ed99c03cde0 None admin] PATH_INFO: `/v3/projects/d861cd3fc0a047d6bf06a6250256ee77` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:11.397116 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1e633d-318d-495c-9990-3ed99c03cde0 None admin] RBAC: Authorizing `identity:get_project(project_id=d861cd3fc0a047d6bf06a6250256ee77)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:11.401032 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1e633d-318d-495c-9990-3ed99c03cde0 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:11.405119 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 114/224] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:32:11 2022] GET /identity/v3/projects/d861cd3fc0a047d6bf06a6250256ee77 => generated 302 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:11.415406 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e633d-318d-495c-9990-3ed99c03cde0 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:11.529343 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:11.533332 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55797d8-e76a-49c4-9233-de33b8b44434 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:11.602135 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e633d-318d-495c-9990-3ed99c03cde0 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:11.620177 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191f84f-5ca9-420c-b5ae-92c1072985d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:11.621698 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191f84f-5ca9-420c-b5ae-92c1072985d4 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:11.622209 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191f84f-5ca9-420c-b5ae-92c1072985d4 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:11.623492 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191f84f-5ca9-420c-b5ae-92c1072985d4 None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:11.625051 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7191f84f-5ca9-420c-b5ae-92c1072985d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:11.625599 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7191f84f-5ca9-420c-b5ae-92c1072985d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:11.628033 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7191f84f-5ca9-420c-b5ae-92c1072985d4 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:11.641849 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 111/225] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:32:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:32:11.647194 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191f84f-5ca9-420c-b5ae-92c1072985d4 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:11.681881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674d8e6-5ed7-44b0-9f65-a3ff871cef88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:11.683817 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674d8e6-5ed7-44b0-9f65-a3ff871cef88 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:11.684534 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674d8e6-5ed7-44b0-9f65-a3ff871cef88 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:11.685360 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674d8e6-5ed7-44b0-9f65-a3ff871cef88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:11.687144 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c674d8e6-5ed7-44b0-9f65-a3ff871cef88 None admin] RBAC: Adding query filter params (role.id=83efac3b92eb44e783f4924c9d9ee842, scope.project.id=d861cd3fc0a047d6bf06a6250256ee77, user.id=a97426e846c0416387b714de740cd043) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:11.687970 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c674d8e6-5ed7-44b0-9f65-a3ff871cef88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:11.690979 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c674d8e6-5ed7-44b0-9f65-a3ff871cef88 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:11.704511 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 115/226] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:32:11 2022] GET /identity/v3/role_assignments?user.id=a97426e846c0416387b714de740cd043&scope.project.id=d861cd3fc0a047d6bf06a6250256ee77&role.id=83efac3b92eb44e783f4924c9d9ee842 => generated 259 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:11.713352 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674d8e6-5ed7-44b0-9f65-a3ff871cef88 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:11.771414 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:11.790309 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191f84f-5ca9-420c-b5ae-92c1072985d4 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:11.886428 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e673c30f-1912-4744-8adc-91161dd101f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:11.887147 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673c30f-1912-4744-8adc-91161dd101f3 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:11.887434 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673c30f-1912-4744-8adc-91161dd101f3 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:11.887720 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673c30f-1912-4744-8adc-91161dd101f3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:11.916625 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e673c30f-1912-4744-8adc-91161dd101f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:11.922899 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-e673c30f-1912-4744-8adc-91161dd101f3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 02:32:11.922899 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:11.922899 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:11.922899 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:11.922899 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:11.922899 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:11.922899 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:11.922899 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:11.922899 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:11.922899 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:11.922899 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:11.922899 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:11.922899 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:11.922899 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:11.922899 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:11.922899 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:11.924636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:11.924636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:11.924636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:11.924636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:11.924636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:11.924636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:11.924636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:11.924636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:11.924636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:11.924636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:11.924636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:11.924636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:11.924636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:11.924636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:11.924636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:11.924636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:11.926704 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:11.926704 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:11.926704 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:11.926704 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:11.926704 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:11.926704 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:11.926704 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:11.926704 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:11.926704 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:11.926704 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:11.926704 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 02:32:11.926704 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:32:11.934305 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674d8e6-5ed7-44b0-9f65-a3ff871cef88 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:11.934778 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 112/227] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Sep 22 02:32:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:11.935778 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e673c30f-1912-4744-8adc-91161dd101f3 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:11.959236 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:12.005765 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b880f0a-76c2-4835-83a9-edf98287c8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:12.006488 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b880f0a-76c2-4835-83a9-edf98287c8b8 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:12.006780 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b880f0a-76c2-4835-83a9-edf98287c8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:12.007116 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b880f0a-76c2-4835-83a9-edf98287c8b8 None admin] PATH_INFO: `/v3/users/142866c87db6452c8fff5c8d3c4aa04e` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:12.008491 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b880f0a-76c2-4835-83a9-edf98287c8b8 None admin] RBAC: Authorizing `identity:get_user(user_id=142866c87db6452c8fff5c8d3c4aa04e)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:12.010021 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b880f0a-76c2-4835-83a9-edf98287c8b8 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:12.011869 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 116/228] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:32:11 2022] GET /identity/v3/users/142866c87db6452c8fff5c8d3c4aa04e => generated 283 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:12.021330 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b880f0a-76c2-4835-83a9-edf98287c8b8 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:12.177369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b880f0a-76c2-4835-83a9-edf98287c8b8 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:12.194283 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e673c30f-1912-4744-8adc-91161dd101f3 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:12.222944 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7685217-634f-480e-b867-dbab083299c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:12.222944 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7685217-634f-480e-b867-dbab083299c1 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:12.222944 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7685217-634f-480e-b867-dbab083299c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:12.222944 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7685217-634f-480e-b867-dbab083299c1 None admin] PATH_INFO: `/v3/projects/d861cd3fc0a047d6bf06a6250256ee77` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:12.227787 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7685217-634f-480e-b867-dbab083299c1 None admin] RBAC: Authorizing `identity:get_project(project_id=d861cd3fc0a047d6bf06a6250256ee77)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:12.227787 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7685217-634f-480e-b867-dbab083299c1 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:12.244767 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 117/229] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:32:12 2022] GET /identity/v3/projects/d861cd3fc0a047d6bf06a6250256ee77 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:12.269189 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070e277-3dbe-429c-8b39-8adfe580030f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:12.270162 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7685217-634f-480e-b867-dbab083299c1 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:12.281815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070e277-3dbe-429c-8b39-8adfe580030f None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:12.282224 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070e277-3dbe-429c-8b39-8adfe580030f None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:12.282405 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070e277-3dbe-429c-8b39-8adfe580030f None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:12.283092 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d070e277-3dbe-429c-8b39-8adfe580030f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:12.285904 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d070e277-3dbe-429c-8b39-8adfe580030f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:12.287367 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d070e277-3dbe-429c-8b39-8adfe580030f None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:12.310252 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 113/230] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:32:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:12.316211 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070e277-3dbe-429c-8b39-8adfe580030f None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:12.350418 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:12.371355 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:12.466803 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070e277-3dbe-429c-8b39-8adfe580030f None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:12.513903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83645fd-5570-44b1-8fd6-5610a93282a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:12.514754 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83645fd-5570-44b1-8fd6-5610a93282a9 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:12.515071 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83645fd-5570-44b1-8fd6-5610a93282a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:12.515397 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83645fd-5570-44b1-8fd6-5610a93282a9 None admin] PATH_INFO: `/v3/users/142866c87db6452c8fff5c8d3c4aa04e` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:12.517610 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83645fd-5570-44b1-8fd6-5610a93282a9 None admin] RBAC: Authorizing `identity:get_user(user_id=142866c87db6452c8fff5c8d3c4aa04e)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:12.519123 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83645fd-5570-44b1-8fd6-5610a93282a9 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:12.521087 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 114/231] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:32:12 2022] GET /identity/v3/users/142866c87db6452c8fff5c8d3c4aa04e => generated 283 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:12.527588 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83645fd-5570-44b1-8fd6-5610a93282a9 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:12.540484 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7685217-634f-480e-b867-dbab083299c1 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:12.668812 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:12.686254 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99a45e-493a-4c5b-bd8f-761104c8f238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:12.687912 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99a45e-493a-4c5b-bd8f-761104c8f238 None admin] REQUEST_METHOD: `PUT` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:12.688724 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83645fd-5570-44b1-8fd6-5610a93282a9 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:12.689013 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99a45e-493a-4c5b-bd8f-761104c8f238 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:12.689788 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99a45e-493a-4c5b-bd8f-761104c8f238 None admin] PATH_INFO: `/v3/projects/d861cd3fc0a047d6bf06a6250256ee77/users/142866c87db6452c8fff5c8d3c4aa04e/roles/31a687548f55493b8ea666fe7e350303` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:12.701318 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99a45e-493a-4c5b-bd8f-761104c8f238 None admin] RBAC: Authorizing `identity:create_grant(project_id=d861cd3fc0a047d6bf06a6250256ee77, user_id=142866c87db6452c8fff5c8d3c4aa04e, role_id=31a687548f55493b8ea666fe7e350303)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:12.704740 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99a45e-493a-4c5b-bd8f-761104c8f238 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:12.731015 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ae019-2ccc-43bc-9102-9ed6814dd898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:12.731709 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ae019-2ccc-43bc-9102-9ed6814dd898 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:12.731998 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ae019-2ccc-43bc-9102-9ed6814dd898 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:12.732362 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ae019-2ccc-43bc-9102-9ed6814dd898 None admin] PATH_INFO: `/v3/projects/d861cd3fc0a047d6bf06a6250256ee77` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:12.734079 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523ae019-2ccc-43bc-9102-9ed6814dd898 None admin] RBAC: Authorizing `identity:get_project(project_id=d861cd3fc0a047d6bf06a6250256ee77)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:12.735772 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523ae019-2ccc-43bc-9102-9ed6814dd898 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:12.738207 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 115/232] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:32:12 2022] GET /identity/v3/projects/d861cd3fc0a047d6bf06a6250256ee77 => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:12.744560 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ae019-2ccc-43bc-9102-9ed6814dd898 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:12.756073 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 118/233] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Sep 22 02:32:12 2022] PUT /identity/v3/projects/d861cd3fc0a047d6bf06a6250256ee77/users/142866c87db6452c8fff5c8d3c4aa04e/roles/31a687548f55493b8ea666fe7e350303 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 02:32:13.038978 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ae019-2ccc-43bc-9102-9ed6814dd898 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:13.119749 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8dab7e-7227-42a2-ab46-4905fe28e823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:13.120835 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8dab7e-7227-42a2-ab46-4905fe28e823 None admin] REQUEST_METHOD: `PUT` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:13.121272 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8dab7e-7227-42a2-ab46-4905fe28e823 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:13.121705 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8dab7e-7227-42a2-ab46-4905fe28e823 None admin] PATH_INFO: `/v3/projects/d861cd3fc0a047d6bf06a6250256ee77/users/142866c87db6452c8fff5c8d3c4aa04e/roles/adec3004b56243fa923bd9f0082a2728` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:13.129088 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8dab7e-7227-42a2-ab46-4905fe28e823 None admin] RBAC: Authorizing `identity:create_grant(project_id=d861cd3fc0a047d6bf06a6250256ee77, user_id=142866c87db6452c8fff5c8d3c4aa04e, role_id=adec3004b56243fa923bd9f0082a2728)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:13.132706 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8dab7e-7227-42a2-ab46-4905fe28e823 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:13.165246 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 116/234] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Sep 22 02:32:12 2022] PUT /identity/v3/projects/d861cd3fc0a047d6bf06a6250256ee77/users/142866c87db6452c8fff5c8d3c4aa04e/roles/adec3004b56243fa923bd9f0082a2728 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 02:32:13.357445 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7341227-d5dd-4fc0-9182-0c176bfb2b48 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:13.357946 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7341227-d5dd-4fc0-9182-0c176bfb2b48 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:13.358363 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7341227-d5dd-4fc0-9182-0c176bfb2b48 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:13.361292 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 119/235] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:13.368760 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3306a48-fc65-42fb-b88d-e00f6acb71a3 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:13.369623 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3306a48-fc65-42fb-b88d-e00f6acb71a3 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:13.370105 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3306a48-fc65-42fb-b88d-e00f6acb71a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:13.401116 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-a3306a48-fc65-42fb-b88d-e00f6acb71a3 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:13.402686 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-a3306a48-fc65-42fb-b88d-e00f6acb71a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:13.412466 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:13.514992 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 117/236] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:32:13.980161 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caddd9a8-b973-403a-a569-d0b18532ba81 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:13.981709 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caddd9a8-b973-403a-a569-d0b18532ba81 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:13.982471 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caddd9a8-b973-403a-a569-d0b18532ba81 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:13.985264 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 120/237] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:13.994717 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0623f30-375c-4450-bcb0-715210f1217c None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:13.995708 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0623f30-375c-4450-bcb0-715210f1217c None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:13.996718 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0623f30-375c-4450-bcb0-715210f1217c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:14.031809 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caddd9a8-b973-403a-a569-d0b18532ba81 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:14.060614 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-b0623f30-375c-4450-bcb0-715210f1217c None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:14.062674 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-b0623f30-375c-4450-bcb0-715210f1217c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:14.238490 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caddd9a8-b973-403a-a569-d0b18532ba81 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:14.259029 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:14.274377 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 118/238] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:32:14.338460 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1119c6-923e-4820-8b58-845476ad1c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:14.339787 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1119c6-923e-4820-8b58-845476ad1c91 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:14.340286 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1119c6-923e-4820-8b58-845476ad1c91 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:14.340789 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1119c6-923e-4820-8b58-845476ad1c91 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:14.346394 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1119c6-923e-4820-8b58-845476ad1c91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:14.349283 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1119c6-923e-4820-8b58-845476ad1c91 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:14.352391 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 121/239] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:32:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:32:14.361093 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0623f30-375c-4450-bcb0-715210f1217c None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:14.484307 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:14.537068 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0623f30-375c-4450-bcb0-715210f1217c None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:14.587500 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f5befb-e616-4a3b-bb96-a8a5a10eaaf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:14.588141 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f5befb-e616-4a3b-bb96-a8a5a10eaaf5 None admin] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:14.588427 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f5befb-e616-4a3b-bb96-a8a5a10eaaf5 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:14.588755 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f5befb-e616-4a3b-bb96-a8a5a10eaaf5 None admin] PATH_INFO: `/v3/users` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:14.589436 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f5befb-e616-4a3b-bb96-a8a5a10eaaf5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:14.590982 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f5befb-e616-4a3b-bb96-a8a5a10eaaf5 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:14.621391 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 119/240] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Sep 22 02:32:14 2022] POST /identity/v3/users => generated 293 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:32:14.800044 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1119c6-923e-4820-8b58-845476ad1c91 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:14.954244 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1119c6-923e-4820-8b58-845476ad1c91 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:15.005403 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89868ab-852b-4753-8062-d0bd919b4196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:15.006342 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89868ab-852b-4753-8062-d0bd919b4196 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:15.006465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89868ab-852b-4753-8062-d0bd919b4196 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:15.006873 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89868ab-852b-4753-8062-d0bd919b4196 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:15.028354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89868ab-852b-4753-8062-d0bd919b4196 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:15.034754 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-d89868ab-852b-4753-8062-d0bd919b4196 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 02:32:15.034754 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:15.034754 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:15.034754 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:15.034754 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:15.034754 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:15.034754 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:15.034754 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:15.034754 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:15.034754 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:15.034754 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:15.034754 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:15.034754 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:15.034754 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:15.034754 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:15.034754 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:15.035734 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:15.035734 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:15.035734 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:15.035734 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:15.035734 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:15.035734 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:15.035734 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:15.035734 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:15.035734 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:15.035734 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:15.035734 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:15.035734 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:15.035734 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:15.035734 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:15.035734 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:15.035734 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:15.036562 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:15.036562 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:15.036562 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:15.036562 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:15.036562 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:15.036562 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:15.036562 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:15.036562 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:15.036562 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:15.036562 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:15.036562 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 02:32:15.036562 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:32:15.039137 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 122/241] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:32:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:15.046288 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f5befb-e616-4a3b-bb96-a8a5a10eaaf5 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:15.259982 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f5befb-e616-4a3b-bb96-a8a5a10eaaf5 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:15.315388 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:15.334813 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7a3288-1064-421d-87d4-6a4d700af2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:15.336262 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a3288-1064-421d-87d4-6a4d700af2e6 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:15.336857 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a3288-1064-421d-87d4-6a4d700af2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:15.337366 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a3288-1064-421d-87d4-6a4d700af2e6 None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:15.338624 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7a3288-1064-421d-87d4-6a4d700af2e6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:15.339259 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7a3288-1064-421d-87d4-6a4d700af2e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:15.341970 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7a3288-1064-421d-87d4-6a4d700af2e6 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:15.365351 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 120/242] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Sep 22 02:32:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:15.371984 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89868ab-852b-4753-8062-d0bd919b4196 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:15.447798 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:15.599662 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89868ab-852b-4753-8062-d0bd919b4196 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:15.679354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a6785-90ed-4bac-8752-32727fbf9866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:15.680501 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a6785-90ed-4bac-8752-32727fbf9866 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:15.681018 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a6785-90ed-4bac-8752-32727fbf9866 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:15.681527 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a6785-90ed-4bac-8752-32727fbf9866 None admin] PATH_INFO: `/v3/users/33130aa187974218bf1b30179a4371e1` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:15.713192 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0a6785-90ed-4bac-8752-32727fbf9866 None admin] RBAC: Authorizing `identity:get_user(user_id=33130aa187974218bf1b30179a4371e1)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:15.716810 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0a6785-90ed-4bac-8752-32727fbf9866 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:15.720911 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 123/243] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:32:15 2022] GET /identity/v3/users/33130aa187974218bf1b30179a4371e1 => generated 297 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:15.727066 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7a3288-1064-421d-87d4-6a4d700af2e6 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:15.901813 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057056a-d228-4acb-b766-5afd19698e9e None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:15.902794 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057056a-d228-4acb-b766-5afd19698e9e None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:15.903656 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057056a-d228-4acb-b766-5afd19698e9e None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:15.907617 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 124/244] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:15.912576 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d0262-50d1-41bd-ad22-94db12c6fb2b None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:15.913207 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d0262-50d1-41bd-ad22-94db12c6fb2b None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:15.913680 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d0262-50d1-41bd-ad22-94db12c6fb2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:15.948746 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-7d7d0262-50d1-41bd-ad22-94db12c6fb2b None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:15.950592 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-7d7d0262-50d1-41bd-ad22-94db12c6fb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:15.969226 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7a3288-1064-421d-87d4-6a4d700af2e6 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:16.062094 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 125/245] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:32:16.080167 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecc290e-662c-496f-9ca4-547ad1ceea4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:16.081314 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc290e-662c-496f-9ca4-547ad1ceea4c None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:16.081827 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc290e-662c-496f-9ca4-547ad1ceea4c None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:16.082282 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc290e-662c-496f-9ca4-547ad1ceea4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:16.265793 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:16.296922 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecc290e-662c-496f-9ca4-547ad1ceea4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:16.299749 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecc290e-662c-496f-9ca4-547ad1ceea4c None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:16.464544 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b4028d-f366-4e89-b1a3-c5db3043fb63 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:16.465210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b4028d-f366-4e89-b1a3-c5db3043fb63 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:16.465793 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b4028d-f366-4e89-b1a3-c5db3043fb63 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:16.467529 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 126/246] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:32:16.536481 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487bb4c-49b2-4d41-9312-7812d0c32870 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:16.537081 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487bb4c-49b2-4d41-9312-7812d0c32870 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:16.537559 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487bb4c-49b2-4d41-9312-7812d0c32870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:16.552194 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 121/247] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:32:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 826 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 02:32:16.561414 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecc290e-662c-496f-9ca4-547ad1ceea4c None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:16.576071 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-1487bb4c-49b2-4d41-9312-7812d0c32870 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:16.577668 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-1487bb4c-49b2-4d41-9312-7812d0c32870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:16.606778 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:16.789611 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 127/248] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 02:32:16.796595 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487bb4c-49b2-4d41-9312-7812d0c32870 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:16.872122 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:16.898476 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecc290e-662c-496f-9ca4-547ad1ceea4c None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:16.997534 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9842a742-b785-4f19-b986-74690e509116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:16.999010 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9842a742-b785-4f19-b986-74690e509116 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:16.999759 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9842a742-b785-4f19-b986-74690e509116 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:17.000467 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9842a742-b785-4f19-b986-74690e509116 None admin] PATH_INFO: `/v3/projects/d861cd3fc0a047d6bf06a6250256ee77` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:17.004358 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9842a742-b785-4f19-b986-74690e509116 None admin] RBAC: Authorizing `identity:get_project(project_id=d861cd3fc0a047d6bf06a6250256ee77)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:17.008566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9842a742-b785-4f19-b986-74690e509116 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:17.012895 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 122/249] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:32:16 2022] GET /identity/v3/projects/d861cd3fc0a047d6bf06a6250256ee77 => generated 302 bytes in 453 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:32:17.022036 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9842a742-b785-4f19-b986-74690e509116 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:17.075664 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487bb4c-49b2-4d41-9312-7812d0c32870 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:17.116930 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788956f2-5c31-483f-83de-c7ac09389153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:17.117899 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788956f2-5c31-483f-83de-c7ac09389153 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:17.118393 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788956f2-5c31-483f-83de-c7ac09389153 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:17.118930 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788956f2-5c31-483f-83de-c7ac09389153 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:17.132659 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788956f2-5c31-483f-83de-c7ac09389153 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:17.138959 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-788956f2-5c31-483f-83de-c7ac09389153 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 02:32:17.138959 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:17.138959 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:17.138959 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:17.138959 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:17.138959 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:17.138959 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:17.138959 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:17.138959 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:17.138959 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:17.138959 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:17.138959 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:17.138959 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:17.138959 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:17.138959 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:17.138959 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:17.139909 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:17.139909 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:17.139909 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:17.139909 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:17.139909 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:17.139909 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:17.139909 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:17.139909 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:17.139909 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:17.139909 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:17.139909 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:17.139909 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:17.139909 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:17.139909 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:17.139909 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:17.139909 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:17.140704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:17.140704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:17.140704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:17.140704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:17.140704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:17.140704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:17.140704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:17.140704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:17.140704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:17.140704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:17.140704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 02:32:17.140704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:32:17.143349 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 128/250] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:32:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:17.144831 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01177027-233f-4a35-ae16-99a79f647c60 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:17.145599 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01177027-233f-4a35-ae16-99a79f647c60 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:17.146056 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01177027-233f-4a35-ae16-99a79f647c60 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:17.147894 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 129/251] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:17.153688 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01177027-233f-4a35-ae16-99a79f647c60 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:17.183260 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:17.298877 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01177027-233f-4a35-ae16-99a79f647c60 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:17.341589 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca003bc-10b4-44cf-ac6a-161d05b09143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:17.342736 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca003bc-10b4-44cf-ac6a-161d05b09143 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:17.343230 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca003bc-10b4-44cf-ac6a-161d05b09143 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:17.343709 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca003bc-10b4-44cf-ac6a-161d05b09143 None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:17.344566 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca003bc-10b4-44cf-ac6a-161d05b09143 None admin] RBAC: Adding query filter params (name=member) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:17.345056 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca003bc-10b4-44cf-ac6a-161d05b09143 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:17.346668 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca003bc-10b4-44cf-ac6a-161d05b09143 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:17.358642 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 130/252] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Sep 22 02:32:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:17.360423 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44f1dd-5462-40c5-a203-9f7f4632e686 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:17.360897 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44f1dd-5462-40c5-a203-9f7f4632e686 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:17.361416 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44f1dd-5462-40c5-a203-9f7f4632e686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:17.379918 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9842a742-b785-4f19-b986-74690e509116 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:17.417170 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-da44f1dd-5462-40c5-a203-9f7f4632e686 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:17.419184 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-da44f1dd-5462-40c5-a203-9f7f4632e686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:17.460838 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff356b-9b46-4884-bb52-2d079ed59594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:17.461868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff356b-9b46-4884-bb52-2d079ed59594 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:17.462349 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff356b-9b46-4884-bb52-2d079ed59594 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:17.462794 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff356b-9b46-4884-bb52-2d079ed59594 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:17.464689 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aff356b-9b46-4884-bb52-2d079ed59594 None admin] RBAC: Adding query filter params (role.id=13d7b9e519274c8bb245c8218571dee8, scope.project.id=d861cd3fc0a047d6bf06a6250256ee77, user.id=33130aa187974218bf1b30179a4371e1) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:17.465197 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aff356b-9b46-4884-bb52-2d079ed59594 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:17.467846 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aff356b-9b46-4884-bb52-2d079ed59594 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:17.478855 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 123/253] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:32:17 2022] GET /identity/v3/role_assignments?user.id=33130aa187974218bf1b30179a4371e1&scope.project.id=d861cd3fc0a047d6bf06a6250256ee77&role.id=13d7b9e519274c8bb245c8218571dee8 => generated 259 bytes in 458 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:17.480892 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff356b-9b46-4884-bb52-2d079ed59594 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:17.535545 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:17.591718 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 131/254] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:32:17.593308 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44f1dd-5462-40c5-a203-9f7f4632e686 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:17.605982 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:17.707913 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff356b-9b46-4884-bb52-2d079ed59594 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:17.808484 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff9def2-1bcc-4430-a5bc-534652c3e447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:17.810599 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9def2-1bcc-4430-a5bc-534652c3e447 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:17.811411 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9def2-1bcc-4430-a5bc-534652c3e447 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:17.812259 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9def2-1bcc-4430-a5bc-534652c3e447 None admin] PATH_INFO: `/v3/users/a97426e846c0416387b714de740cd043` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:17.814794 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff9def2-1bcc-4430-a5bc-534652c3e447 None admin] RBAC: Authorizing `identity:get_user(user_id=a97426e846c0416387b714de740cd043)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:17.818284 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff9def2-1bcc-4430-a5bc-534652c3e447 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:17.826963 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 124/255] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:32:17 2022] GET /identity/v3/users/a97426e846c0416387b714de740cd043 => generated 297 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:17.832025 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff9def2-1bcc-4430-a5bc-534652c3e447 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:17.871265 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44f1dd-5462-40c5-a203-9f7f4632e686 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:17.916098 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eebecf-a32d-43be-8882-37795c251a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:17.917328 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eebecf-a32d-43be-8882-37795c251a95 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:17.917607 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eebecf-a32d-43be-8882-37795c251a95 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:17.917883 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eebecf-a32d-43be-8882-37795c251a95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:17.929998 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eebecf-a32d-43be-8882-37795c251a95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:17.935703 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-11eebecf-a32d-43be-8882-37795c251a95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:32:17.935703 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:17.935703 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:17.935703 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:17.935703 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:17.935703 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:17.935703 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:17.935703 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:17.935703 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:17.935703 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:17.935703 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:17.935703 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:17.935703 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:17.935703 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:17.935703 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:17.935703 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:17.937384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:17.937384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:17.937384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:17.937384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:17.937384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:17.937384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:17.937384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:17.937384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:17.937384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:17.937384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:17.937384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:17.937384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:17.937384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:17.937384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:17.937384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:17.937384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:17.938964 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:17.938964 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:17.938964 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:17.938964 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:17.938964 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:17.938964 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:17.938964 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:17.938964 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:17.938964 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:17.938964 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:17.938964 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:32:17.938964 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:32:17.938964 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 132/256] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Sep 22 02:32:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:17.961414 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eebecf-a32d-43be-8882-37795c251a95 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:18.044908 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:18.076502 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:18.087938 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eebecf-a32d-43be-8882-37795c251a95 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:18.105806 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff9def2-1bcc-4430-a5bc-534652c3e447 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:18.130985 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c53da5-ec9f-4e44-9236-c7078dea768e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:18.130985 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c53da5-ec9f-4e44-9236-c7078dea768e None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:18.130985 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c53da5-ec9f-4e44-9236-c7078dea768e None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:18.130985 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c53da5-ec9f-4e44-9236-c7078dea768e None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:18.131656 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c53da5-ec9f-4e44-9236-c7078dea768e None admin] RBAC: Adding query filter params (name=admin) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:18.131656 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c53da5-ec9f-4e44-9236-c7078dea768e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:18.131656 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c53da5-ec9f-4e44-9236-c7078dea768e None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:18.141368 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 133/257] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:32:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:32:18.147041 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c53da5-ec9f-4e44-9236-c7078dea768e None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:18.214465 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f99bb-19b1-4e8b-8a23-0bc187bbf76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:18.215902 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f99bb-19b1-4e8b-8a23-0bc187bbf76f None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:18.216580 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f99bb-19b1-4e8b-8a23-0bc187bbf76f None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:18.217401 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f99bb-19b1-4e8b-8a23-0bc187bbf76f None admin] PATH_INFO: `/v3/projects/d861cd3fc0a047d6bf06a6250256ee77` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:18.223197 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f99bb-19b1-4e8b-8a23-0bc187bbf76f None admin] RBAC: Authorizing `identity:get_project(project_id=d861cd3fc0a047d6bf06a6250256ee77)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:18.226526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f99bb-19b1-4e8b-8a23-0bc187bbf76f None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:18.239596 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 125/258] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:32:17 2022] GET /identity/v3/projects/d861cd3fc0a047d6bf06a6250256ee77 => generated 302 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:18.241872 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f99bb-19b1-4e8b-8a23-0bc187bbf76f None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:18.264679 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c53da5-ec9f-4e44-9236-c7078dea768e None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:18.296365 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:18.307781 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d58bf5-c398-40c1-9ba3-167f13796eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:18.308617 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d58bf5-c398-40c1-9ba3-167f13796eba None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:18.308998 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d58bf5-c398-40c1-9ba3-167f13796eba None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:18.309376 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d58bf5-c398-40c1-9ba3-167f13796eba None admin] PATH_INFO: `/v3/users/142866c87db6452c8fff5c8d3c4aa04e` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:18.310748 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d58bf5-c398-40c1-9ba3-167f13796eba None admin] RBAC: Authorizing `identity:get_user(user_id=142866c87db6452c8fff5c8d3c4aa04e)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:18.312368 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d58bf5-c398-40c1-9ba3-167f13796eba None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:18.314672 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 134/259] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:32:18 2022] GET /identity/v3/users/142866c87db6452c8fff5c8d3c4aa04e => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:32:18.315504 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d58bf5-c398-40c1-9ba3-167f13796eba None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:18.443210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d58bf5-c398-40c1-9ba3-167f13796eba None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:18.476827 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f99bb-19b1-4e8b-8a23-0bc187bbf76f None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:18.516269 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8710107e-83b8-4a1c-8d7a-5e35e057477d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:18.516975 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8710107e-83b8-4a1c-8d7a-5e35e057477d None admin] REQUEST_METHOD: `PUT` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:18.517252 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8710107e-83b8-4a1c-8d7a-5e35e057477d None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:18.517531 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8710107e-83b8-4a1c-8d7a-5e35e057477d None admin] PATH_INFO: `/v3/projects/d861cd3fc0a047d6bf06a6250256ee77/users/a97426e846c0416387b714de740cd043/roles/83efac3b92eb44e783f4924c9d9ee842` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:18.524467 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8710107e-83b8-4a1c-8d7a-5e35e057477d None admin] RBAC: Authorizing `identity:create_grant(project_id=d861cd3fc0a047d6bf06a6250256ee77, user_id=a97426e846c0416387b714de740cd043, role_id=83efac3b92eb44e783f4924c9d9ee842)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:18.526302 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8710107e-83b8-4a1c-8d7a-5e35e057477d None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:18.533254 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588bbd9d-89bf-4222-9b74-f6d2149b0157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:18.533917 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bbd9d-89bf-4222-9b74-f6d2149b0157 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:18.534249 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bbd9d-89bf-4222-9b74-f6d2149b0157 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:18.534528 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bbd9d-89bf-4222-9b74-f6d2149b0157 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:18.548529 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588bbd9d-89bf-4222-9b74-f6d2149b0157 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:18.555103 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-588bbd9d-89bf-4222-9b74-f6d2149b0157 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 02:32:18.555103 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:18.555103 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:18.555103 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:18.555103 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:18.555103 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:18.555103 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:18.555103 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:18.555103 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:18.555103 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:18.555103 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:18.555103 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:18.555103 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:18.555103 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:18.555103 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:18.555103 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:18.566429 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:18.566429 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:18.566429 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:18.566429 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:18.566429 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:18.566429 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:18.566429 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:18.566429 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:18.566429 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:18.566429 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:18.566429 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:18.566429 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:18.566429 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:18.566429 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:18.566429 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:18.566429 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:18.580984 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:18.580984 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:18.580984 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:18.580984 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:18.580984 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:18.580984 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:18.580984 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:18.580984 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:18.580984 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:18.580984 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:18.580984 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 02:32:18.580984 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:32:18.583669 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 135/260] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Sep 22 02:32:18 2022] PUT /identity/v3/projects/d861cd3fc0a047d6bf06a6250256ee77/users/a97426e846c0416387b714de740cd043/roles/83efac3b92eb44e783f4924c9d9ee842 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 02:32:18.583669 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8710107e-83b8-4a1c-8d7a-5e35e057477d None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:18.584133 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 126/261] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Sep 22 02:32:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:18.584133 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588bbd9d-89bf-4222-9b74-f6d2149b0157 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:18.638996 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:18.782986 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:18.810110 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588bbd9d-89bf-4222-9b74-f6d2149b0157 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:18.892519 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8710107e-83b8-4a1c-8d7a-5e35e057477d None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:18.939125 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72189f-cf09-4805-a581-bd0d5ca53841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:18.939780 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72189f-cf09-4805-a581-bd0d5ca53841 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:18.940068 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72189f-cf09-4805-a581-bd0d5ca53841 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:18.940339 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72189f-cf09-4805-a581-bd0d5ca53841 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:18.942050 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99027178-55ca-4c6f-8c69-89716b9b797d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:18.943830 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99027178-55ca-4c6f-8c69-89716b9b797d None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:18.944728 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99027178-55ca-4c6f-8c69-89716b9b797d None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:18.945483 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99027178-55ca-4c6f-8c69-89716b9b797d None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:18.947036 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99027178-55ca-4c6f-8c69-89716b9b797d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:18.948416 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99027178-55ca-4c6f-8c69-89716b9b797d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:18.951740 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99027178-55ca-4c6f-8c69-89716b9b797d None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:18.969499 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 127/262] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:32:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:18.972523 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8964e1-7f5c-450d-a1a1-95e7c2757337 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:18.973349 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8964e1-7f5c-450d-a1a1-95e7c2757337 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:18.974345 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8964e1-7f5c-450d-a1a1-95e7c2757337 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:18.979182 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 128/263] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:18.981020 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8964e1-7f5c-450d-a1a1-95e7c2757337 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:19.029595 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d72189f-cf09-4805-a581-bd0d5ca53841 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:19.031619 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d72189f-cf09-4805-a581-bd0d5ca53841 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:19.104866 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:19.148298 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:19.180127 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8964e1-7f5c-450d-a1a1-95e7c2757337 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:19.195539 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 136/264] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:32:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 639 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 02:32:19.198631 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb67075-d38e-4e3d-a5d4-675d93c052d7 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:19.199530 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb67075-d38e-4e3d-a5d4-675d93c052d7 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:19.200276 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb67075-d38e-4e3d-a5d4-675d93c052d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:19.222692 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db1b54-69d4-4248-910e-afbaf31a44dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:19.224004 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db1b54-69d4-4248-910e-afbaf31a44dd None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:19.224307 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db1b54-69d4-4248-910e-afbaf31a44dd None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:19.224589 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db1b54-69d4-4248-910e-afbaf31a44dd None admin] PATH_INFO: `/v3/users/142866c87db6452c8fff5c8d3c4aa04e` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:19.225867 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db1b54-69d4-4248-910e-afbaf31a44dd None admin] RBAC: Authorizing `identity:get_user(user_id=142866c87db6452c8fff5c8d3c4aa04e)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:19.227419 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db1b54-69d4-4248-910e-afbaf31a44dd None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:19.230081 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 129/265] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:32:18 2022] GET /identity/v3/users/142866c87db6452c8fff5c8d3c4aa04e => generated 283 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:19.230994 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db1b54-69d4-4248-910e-afbaf31a44dd None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:19.235277 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-0cb67075-d38e-4e3d-a5d4-675d93c052d7 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:19.237752 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-0cb67075-d38e-4e3d-a5d4-675d93c052d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:19.371644 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db1b54-69d4-4248-910e-afbaf31a44dd None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:19.373290 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 137/266] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:32:19.375291 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb67075-d38e-4e3d-a5d4-675d93c052d7 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:19.452378 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fad8f0-1f7d-411c-971a-291a4a04e4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:19.453163 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fad8f0-1f7d-411c-971a-291a4a04e4ac None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:19.453328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fad8f0-1f7d-411c-971a-291a4a04e4ac None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:19.453584 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fad8f0-1f7d-411c-971a-291a4a04e4ac None admin] PATH_INFO: `/v3/projects/d861cd3fc0a047d6bf06a6250256ee77` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:19.460473 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fad8f0-1f7d-411c-971a-291a4a04e4ac None admin] RBAC: Authorizing `identity:get_project(project_id=d861cd3fc0a047d6bf06a6250256ee77)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:19.461440 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fad8f0-1f7d-411c-971a-291a4a04e4ac None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:19.464141 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 130/267] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:32:19 2022] GET /identity/v3/projects/d861cd3fc0a047d6bf06a6250256ee77 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:19.465416 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf57e93-02c8-4583-a865-441dcf4503ab None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:19.465714 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf57e93-02c8-4583-a865-441dcf4503ab None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:19.465994 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf57e93-02c8-4583-a865-441dcf4503ab None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:19.467592 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 131/268] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:19.472776 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf57e93-02c8-4583-a865-441dcf4503ab None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:19.504824 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:19.563815 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:19.596913 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb67075-d38e-4e3d-a5d4-675d93c052d7 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:19.675497 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6d363-32dc-460c-8513-3ea0c6bede7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:19.677152 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6d363-32dc-460c-8513-3ea0c6bede7b None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:19.677920 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6d363-32dc-460c-8513-3ea0c6bede7b None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:19.678775 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6d363-32dc-460c-8513-3ea0c6bede7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:19.699606 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf57e93-02c8-4583-a865-441dcf4503ab None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:19.821598 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b2c68-9a54-4d98-9a61-40d39de0654f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:19.822233 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b2c68-9a54-4d98-9a61-40d39de0654f None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:19.822519 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b2c68-9a54-4d98-9a61-40d39de0654f None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:19.822885 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b2c68-9a54-4d98-9a61-40d39de0654f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:19.823771 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87b2c68-9a54-4d98-9a61-40d39de0654f None admin] RBAC: Adding query filter params (role.id=31a687548f55493b8ea666fe7e350303, scope.project.id=d861cd3fc0a047d6bf06a6250256ee77, user.id=142866c87db6452c8fff5c8d3c4aa04e) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:19.824089 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87b2c68-9a54-4d98-9a61-40d39de0654f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:19.825570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87b2c68-9a54-4d98-9a61-40d39de0654f None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:19.832487 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b6d363-32dc-460c-8513-3ea0c6bede7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:19.834949 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 132/269] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:32:19 2022] GET /identity/v3/role_assignments?user.id=142866c87db6452c8fff5c8d3c4aa04e&scope.project.id=d861cd3fc0a047d6bf06a6250256ee77&role.id=31a687548f55493b8ea666fe7e350303 => generated 608 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:19.835201 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b6d363-32dc-460c-8513-3ea0c6bede7b None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:19.836803 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673bc89-3ee8-464f-9437-48f0e1a910d5 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:19.837335 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673bc89-3ee8-464f-9437-48f0e1a910d5 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:19.837816 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673bc89-3ee8-464f-9437-48f0e1a910d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:19.862857 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-e673bc89-3ee8-464f-9437-48f0e1a910d5 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:19.864516 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-e673bc89-3ee8-464f-9437-48f0e1a910d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:19.965756 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 133/270] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:32:19.968407 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e673bc89-3ee8-464f-9437-48f0e1a910d5 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:19.991403 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:20.064632 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:20.106566 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 138/271] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:32:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 732 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 02:32:20.116289 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6d363-32dc-460c-8513-3ea0c6bede7b None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:20.223737 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e673bc89-3ee8-464f-9437-48f0e1a910d5 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:20.297257 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef2d66-7d9d-431c-8ca9-2afa67e2b199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:20.298204 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef2d66-7d9d-431c-8ca9-2afa67e2b199 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:20.298652 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef2d66-7d9d-431c-8ca9-2afa67e2b199 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:20.299095 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef2d66-7d9d-431c-8ca9-2afa67e2b199 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:20.337055 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ef2d66-7d9d-431c-8ca9-2afa67e2b199 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:20.344019 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-b0ef2d66-7d9d-431c-8ca9-2afa67e2b199 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 02:32:20.344019 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:20.344019 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:20.344019 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:20.344019 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:20.344019 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:20.344019 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:20.344019 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:20.344019 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:20.344019 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:20.344019 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:20.344019 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:20.344019 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:20.344019 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:20.344019 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:20.344019 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:20.345158 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:20.345158 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:20.345158 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:20.345158 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:20.345158 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:20.345158 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:20.345158 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:20.345158 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:20.345158 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:20.345158 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:20.345158 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:20.345158 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:20.345158 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:20.345158 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:20.345158 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:20.345158 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:20.346060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:20.346060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:20.346060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:20.346060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:20.346060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:20.346060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:20.346060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:20.346060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:20.346060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:20.346060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:20.346060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 02:32:20.346060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:32:20.351297 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 134/272] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:32:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:20.361335 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef2d66-7d9d-431c-8ca9-2afa67e2b199 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:20.383835 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6d363-32dc-460c-8513-3ea0c6bede7b None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:20.459115 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4d4f5f-b96d-4fa7-8a5b-f79ef994925c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:20.459779 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d4f5f-b96d-4fa7-8a5b-f79ef994925c None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:20.460067 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d4f5f-b96d-4fa7-8a5b-f79ef994925c None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:20.460350 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d4f5f-b96d-4fa7-8a5b-f79ef994925c None admin] PATH_INFO: `/v3/projects/d861cd3fc0a047d6bf06a6250256ee77` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:20.466304 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4d4f5f-b96d-4fa7-8a5b-f79ef994925c None admin] RBAC: Authorizing `identity:get_project(project_id=d861cd3fc0a047d6bf06a6250256ee77)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:20.467845 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4d4f5f-b96d-4fa7-8a5b-f79ef994925c None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:20.469858 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 139/273] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:32:20 2022] GET /identity/v3/projects/d861cd3fc0a047d6bf06a6250256ee77 => generated 302 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:20.477317 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4d4f5f-b96d-4fa7-8a5b-f79ef994925c None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:20.517391 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:20.560182 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef2d66-7d9d-431c-8ca9-2afa67e2b199 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:20.606188 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74770287-a2ca-48d1-a950-ae098efc0034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:20.606891 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74770287-a2ca-48d1-a950-ae098efc0034 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:20.607179 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74770287-a2ca-48d1-a950-ae098efc0034 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:20.607459 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74770287-a2ca-48d1-a950-ae098efc0034 None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:20.608175 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74770287-a2ca-48d1-a950-ae098efc0034 None admin] RBAC: Adding query filter params (name=member) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:20.608485 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74770287-a2ca-48d1-a950-ae098efc0034 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:20.609922 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74770287-a2ca-48d1-a950-ae098efc0034 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:20.624963 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 135/274] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Sep 22 02:32:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:20.641588 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74770287-a2ca-48d1-a950-ae098efc0034 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:20.648456 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:20.781448 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4d4f5f-b96d-4fa7-8a5b-f79ef994925c None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:20.798364 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74770287-a2ca-48d1-a950-ae098efc0034 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:20.833285 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448b94f-4efa-4399-a2ee-4133344eccbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:20.833919 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448b94f-4efa-4399-a2ee-4133344eccbe None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:20.834210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448b94f-4efa-4399-a2ee-4133344eccbe None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:20.834491 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448b94f-4efa-4399-a2ee-4133344eccbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:20.836124 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448b94f-4efa-4399-a2ee-4133344eccbe None admin] RBAC: Adding query filter params (role.id=adec3004b56243fa923bd9f0082a2728, scope.project.id=d861cd3fc0a047d6bf06a6250256ee77, user.id=142866c87db6452c8fff5c8d3c4aa04e) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:20.836449 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448b94f-4efa-4399-a2ee-4133344eccbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:20.839071 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448b94f-4efa-4399-a2ee-4133344eccbe None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:20.848236 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 140/275] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:32:20 2022] GET /identity/v3/role_assignments?user.id=142866c87db6452c8fff5c8d3c4aa04e&scope.project.id=d861cd3fc0a047d6bf06a6250256ee77&role.id=adec3004b56243fa923bd9f0082a2728 => generated 608 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:20.849155 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448b94f-4efa-4399-a2ee-4133344eccbe None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:20.852282 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:20.862250 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9908ce-1fa0-4942-80b6-1cfe9e526bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:20.863469 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9908ce-1fa0-4942-80b6-1cfe9e526bdb None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:20.864090 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9908ce-1fa0-4942-80b6-1cfe9e526bdb None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:20.864685 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9908ce-1fa0-4942-80b6-1cfe9e526bdb None admin] PATH_INFO: `/v3/users/a26448711c4f4807ac055a00470b80eb` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:20.893305 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9908ce-1fa0-4942-80b6-1cfe9e526bdb None admin] RBAC: Authorizing `identity:get_user(user_id=a26448711c4f4807ac055a00470b80eb)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:20.898182 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9908ce-1fa0-4942-80b6-1cfe9e526bdb None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:20.906421 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 136/276] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:32:20 2022] GET /identity/v3/users/a26448711c4f4807ac055a00470b80eb => generated 293 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:32:20.917374 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9908ce-1fa0-4942-80b6-1cfe9e526bdb None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:21.117567 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448b94f-4efa-4399-a2ee-4133344eccbe None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:21.154529 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:21.203102 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61913e97-13b0-4d4d-8e67-63437f6d9a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:21.207077 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61913e97-13b0-4d4d-8e67-63437f6d9a19 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:21.207544 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61913e97-13b0-4d4d-8e67-63437f6d9a19 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:21.208013 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61913e97-13b0-4d4d-8e67-63437f6d9a19 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:21.210129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61913e97-13b0-4d4d-8e67-63437f6d9a19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:21.212806 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61913e97-13b0-4d4d-8e67-63437f6d9a19 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:21.216468 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 141/277] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:32:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:21.223533 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61913e97-13b0-4d4d-8e67-63437f6d9a19 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:21.234362 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9908ce-1fa0-4942-80b6-1cfe9e526bdb None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:21.303747 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b5c18-3d29-493b-ac43-73df14d57b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:21.303747 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b5c18-3d29-493b-ac43-73df14d57b68 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:21.303747 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b5c18-3d29-493b-ac43-73df14d57b68 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:21.303747 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b5c18-3d29-493b-ac43-73df14d57b68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:21.304377 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b5c18-3d29-493b-ac43-73df14d57b68 None admin] RBAC: Adding query filter params (role.id=83efac3b92eb44e783f4924c9d9ee842, user.id=a26448711c4f4807ac055a00470b80eb, scope.system=all) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:21.304377 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b5c18-3d29-493b-ac43-73df14d57b68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:21.304377 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b5c18-3d29-493b-ac43-73df14d57b68 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:21.312301 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 137/278] 10.222.0.20 () {56 vars in 1308 bytes} [Thu Sep 22 02:32:20 2022] GET /identity/v3/role_assignments?user.id=a26448711c4f4807ac055a00470b80eb&scope.system=all&role.id=83efac3b92eb44e783f4924c9d9ee842 => generated 226 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:21.314336 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07a7c7-91ed-4b30-b366-604b1d181ad0 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:21.314671 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07a7c7-91ed-4b30-b366-604b1d181ad0 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:21.314938 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07a7c7-91ed-4b30-b366-604b1d181ad0 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:21.316385 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 138/279] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:21.323615 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51fd39-b9a0-4896-bc52-399c6d92c0a3 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:21.324398 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51fd39-b9a0-4896-bc52-399c6d92c0a3 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:21.324722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51fd39-b9a0-4896-bc52-399c6d92c0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:21.349425 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-8c51fd39-b9a0-4896-bc52-399c6d92c0a3 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:21.350903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-8c51fd39-b9a0-4896-bc52-399c6d92c0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:21.427338 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:21.452162 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61913e97-13b0-4d4d-8e67-63437f6d9a19 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:21.461858 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 139/280] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 02:32:21.497672 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7dbbaa-23dc-4a93-8bd8-98a88f6e9511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:21.498326 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7dbbaa-23dc-4a93-8bd8-98a88f6e9511 None admin] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:21.498614 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7dbbaa-23dc-4a93-8bd8-98a88f6e9511 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:21.498934 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7dbbaa-23dc-4a93-8bd8-98a88f6e9511 None admin] PATH_INFO: `/v3/users` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:21.499622 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7dbbaa-23dc-4a93-8bd8-98a88f6e9511 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:21.501180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7dbbaa-23dc-4a93-8bd8-98a88f6e9511 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:21.529696 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 142/281] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Sep 22 02:32:21 2022] POST /identity/v3/users => generated 293 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:32:21.741088 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51fd39-b9a0-4896-bc52-399c6d92c0a3 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:21.857195 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51fd39-b9a0-4896-bc52-399c6d92c0a3 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:21.896614 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5665677-0b94-4224-a731-781777b51dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:21.897291 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5665677-0b94-4224-a731-781777b51dfd None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:21.897574 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5665677-0b94-4224-a731-781777b51dfd None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:21.897850 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5665677-0b94-4224-a731-781777b51dfd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:21.912579 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5665677-0b94-4224-a731-781777b51dfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:21.918367 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-d5665677-0b94-4224-a731-781777b51dfd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 02:32:21.918367 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:21.918367 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:21.918367 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:21.918367 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:21.918367 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:21.918367 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:21.918367 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:21.918367 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:21.918367 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:21.918367 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:21.918367 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:21.918367 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:21.918367 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:21.918367 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:21.918367 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:21.919335 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:21.919335 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:21.919335 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:21.919335 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:21.919335 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:21.919335 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:21.919335 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:21.919335 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:21.919335 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:21.919335 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:21.919335 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:21.919335 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:21.919335 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:21.919335 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:21.919335 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:21.919335 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:21.920147 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:21.920147 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:21.920147 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:21.920147 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:21.920147 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:21.920147 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:21.920147 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:21.920147 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:21.920147 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:21.920147 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:21.920147 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 02:32:21.920147 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:32:21.920147 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 140/282] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:32:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:21.927392 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7dbbaa-23dc-4a93-8bd8-98a88f6e9511 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:21.993071 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:22.089070 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7dbbaa-23dc-4a93-8bd8-98a88f6e9511 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:22.094697 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54344526-1886-434f-9c51-542d720db952 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:22.095072 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54344526-1886-434f-9c51-542d720db952 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:22.095303 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54344526-1886-434f-9c51-542d720db952 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:22.096968 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 141/283] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:22.103507 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91e5fc-efd5-4b5b-8b57-5e96ab7ec3c1 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:22.104465 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91e5fc-efd5-4b5b-8b57-5e96ab7ec3c1 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:22.104625 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91e5fc-efd5-4b5b-8b57-5e96ab7ec3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:22.139353 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-fd91e5fc-efd5-4b5b-8b57-5e96ab7ec3c1 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:22.140586 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-fd91e5fc-efd5-4b5b-8b57-5e96ab7ec3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:22.174798 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c0ef9-cc50-407e-bf49-4edbe7a181f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:22.175605 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c0ef9-cc50-407e-bf49-4edbe7a181f2 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:22.176031 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c0ef9-cc50-407e-bf49-4edbe7a181f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:22.176178 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c0ef9-cc50-407e-bf49-4edbe7a181f2 None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:22.176900 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8c0ef9-cc50-407e-bf49-4edbe7a181f2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:22.177280 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8c0ef9-cc50-407e-bf49-4edbe7a181f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:22.178735 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8c0ef9-cc50-407e-bf49-4edbe7a181f2 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:22.198031 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 143/284] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Sep 22 02:32:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:22.203959 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c0ef9-cc50-407e-bf49-4edbe7a181f2 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:22.225905 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:22.319475 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c0ef9-cc50-407e-bf49-4edbe7a181f2 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:22.323956 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:22.358153 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 142/285] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 02:32:22.370998 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c78845-d54d-4128-a34e-ba0d7c36071f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:22.371686 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c78845-d54d-4128-a34e-ba0d7c36071f None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:22.371974 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c78845-d54d-4128-a34e-ba0d7c36071f None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:22.372247 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c78845-d54d-4128-a34e-ba0d7c36071f None admin] PATH_INFO: `/v3/users/33130aa187974218bf1b30179a4371e1` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:22.373609 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c78845-d54d-4128-a34e-ba0d7c36071f None admin] RBAC: Authorizing `identity:get_user(user_id=33130aa187974218bf1b30179a4371e1)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:22.375100 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c78845-d54d-4128-a34e-ba0d7c36071f None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:22.377059 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 144/286] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:32:22 2022] GET /identity/v3/users/33130aa187974218bf1b30179a4371e1 => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:22.382757 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91e5fc-efd5-4b5b-8b57-5e96ab7ec3c1 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:22.534548 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91e5fc-efd5-4b5b-8b57-5e96ab7ec3c1 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:22.585982 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5628081-f02e-4c81-878d-70538544b1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:22.586560 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5628081-f02e-4c81-878d-70538544b1e0 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:22.586857 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5628081-f02e-4c81-878d-70538544b1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:22.587142 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5628081-f02e-4c81-878d-70538544b1e0 None admin] PATH_INFO: `/v3/projects/d861cd3fc0a047d6bf06a6250256ee77` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:22.588398 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5628081-f02e-4c81-878d-70538544b1e0 None admin] RBAC: Authorizing `identity:get_project(project_id=d861cd3fc0a047d6bf06a6250256ee77)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:22.589879 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5628081-f02e-4c81-878d-70538544b1e0 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:22.591763 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 143/287] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:32:22 2022] GET /identity/v3/projects/d861cd3fc0a047d6bf06a6250256ee77 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:32:22.600716 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c78845-d54d-4128-a34e-ba0d7c36071f None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:22.720306 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c78845-d54d-4128-a34e-ba0d7c36071f None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:22.756738 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19823b53-7643-4a62-9d34-ad048b97d8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:22.757425 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19823b53-7643-4a62-9d34-ad048b97d8fa None admin] REQUEST_METHOD: `PUT` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:22.757708 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19823b53-7643-4a62-9d34-ad048b97d8fa None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:22.757983 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19823b53-7643-4a62-9d34-ad048b97d8fa None admin] PATH_INFO: `/v3/projects/d861cd3fc0a047d6bf06a6250256ee77/users/33130aa187974218bf1b30179a4371e1/roles/13d7b9e519274c8bb245c8218571dee8` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:22.760038 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19823b53-7643-4a62-9d34-ad048b97d8fa None admin] RBAC: Authorizing `identity:create_grant(project_id=d861cd3fc0a047d6bf06a6250256ee77, user_id=33130aa187974218bf1b30179a4371e1, role_id=13d7b9e519274c8bb245c8218571dee8)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:22.762027 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19823b53-7643-4a62-9d34-ad048b97d8fa None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:22.786020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 145/288] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Sep 22 02:32:22 2022] PUT /identity/v3/projects/d861cd3fc0a047d6bf06a6250256ee77/users/33130aa187974218bf1b30179a4371e1/roles/13d7b9e519274c8bb245c8218571dee8 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 02:32:22.844004 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5628081-f02e-4c81-878d-70538544b1e0 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:22.909978 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:23.017970 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5628081-f02e-4c81-878d-70538544b1e0 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:23.063205 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9fe8ab-ee36-4b89-8501-489f14b2397e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:23.063932 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fe8ab-ee36-4b89-8501-489f14b2397e None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:23.064222 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fe8ab-ee36-4b89-8501-489f14b2397e None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:23.064501 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fe8ab-ee36-4b89-8501-489f14b2397e None admin] PATH_INFO: `/v3/roles/member` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:23.082875 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9fe8ab-ee36-4b89-8501-489f14b2397e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:23.088859 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-7c9fe8ab-ee36-4b89-8501-489f14b2397e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 02:32:23.088859 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:23.088859 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:23.088859 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:23.088859 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:23.088859 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:23.088859 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:23.088859 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:23.088859 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:23.088859 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:23.088859 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:23.088859 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:23.088859 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:23.088859 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:23.088859 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:23.088859 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:23.090729 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:23.090729 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:23.090729 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:23.090729 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:23.090729 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:23.090729 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:23.090729 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:23.090729 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:23.090729 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:23.090729 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:23.090729 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:23.090729 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:23.090729 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:23.090729 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:23.090729 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:23.090729 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:23.092273 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:23.092273 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:23.092273 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:23.092273 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:23.092273 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:23.092273 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:23.092273 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:23.092273 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:23.092273 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:23.092273 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:23.092273 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 02:32:23.092273 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:32:23.096282 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 144/289] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:32:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:23.104851 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19823b53-7643-4a62-9d34-ad048b97d8fa None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:23.157555 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:23.243193 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19823b53-7643-4a62-9d34-ad048b97d8fa None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:23.281282 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f3373-ba3b-4551-a808-800e500afda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:23.281915 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f3373-ba3b-4551-a808-800e500afda1 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:23.282198 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f3373-ba3b-4551-a808-800e500afda1 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:23.282474 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f3373-ba3b-4551-a808-800e500afda1 None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:23.283306 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997f3373-ba3b-4551-a808-800e500afda1 None admin] RBAC: Adding query filter params (name=member) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:23.283619 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997f3373-ba3b-4551-a808-800e500afda1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:23.285296 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997f3373-ba3b-4551-a808-800e500afda1 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:23.295764 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 146/290] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Sep 22 02:32:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:32:23.301576 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9fe8ab-ee36-4b89-8501-489f14b2397e None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:23.444753 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:23.448033 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9fe8ab-ee36-4b89-8501-489f14b2397e None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:23.527792 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14069ea2-4256-4254-8a0b-aaea1b11aa26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:23.529443 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14069ea2-4256-4254-8a0b-aaea1b11aa26 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:23.530276 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14069ea2-4256-4254-8a0b-aaea1b11aa26 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:23.531097 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14069ea2-4256-4254-8a0b-aaea1b11aa26 None admin] PATH_INFO: `/v3/users/a97426e846c0416387b714de740cd043` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:23.533618 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14069ea2-4256-4254-8a0b-aaea1b11aa26 None admin] RBAC: Authorizing `identity:get_user(user_id=a97426e846c0416387b714de740cd043)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:23.536815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14069ea2-4256-4254-8a0b-aaea1b11aa26 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:23.540251 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 145/291] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:32:23 2022] GET /identity/v3/users/a97426e846c0416387b714de740cd043 => generated 297 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:23.546634 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f3373-ba3b-4551-a808-800e500afda1 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:23.682679 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:23.699829 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f3373-ba3b-4551-a808-800e500afda1 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:23.746791 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00681f2b-dfe2-4cef-9110-685d0b170a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:23.747538 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00681f2b-dfe2-4cef-9110-685d0b170a35 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:23.747870 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00681f2b-dfe2-4cef-9110-685d0b170a35 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:23.748154 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00681f2b-dfe2-4cef-9110-685d0b170a35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:23.837251 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00681f2b-dfe2-4cef-9110-685d0b170a35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:23.839382 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00681f2b-dfe2-4cef-9110-685d0b170a35 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:23.943600 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:23.961759 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 147/292] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:32:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 22 02:32:23.967855 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14069ea2-4256-4254-8a0b-aaea1b11aa26 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:24.132615 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14069ea2-4256-4254-8a0b-aaea1b11aa26 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:24.199213 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be81c1-7577-406e-8ce8-81f6abd5c599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:24.199884 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be81c1-7577-406e-8ce8-81f6abd5c599 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:24.200167 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be81c1-7577-406e-8ce8-81f6abd5c599 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:24.200444 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be81c1-7577-406e-8ce8-81f6abd5c599 None admin] PATH_INFO: `/v3/projects/d861cd3fc0a047d6bf06a6250256ee77` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:24.201662 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46be81c1-7577-406e-8ce8-81f6abd5c599 None admin] RBAC: Authorizing `identity:get_project(project_id=d861cd3fc0a047d6bf06a6250256ee77)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:24.203171 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46be81c1-7577-406e-8ce8-81f6abd5c599 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:24.205256 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 146/293] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:32:23 2022] GET /identity/v3/projects/d861cd3fc0a047d6bf06a6250256ee77 => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:24.210974 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00681f2b-dfe2-4cef-9110-685d0b170a35 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:24.371886 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00681f2b-dfe2-4cef-9110-685d0b170a35 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:24.414504 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878891be-3bfd-4c2a-89cc-1abfb4e08003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:24.415162 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878891be-3bfd-4c2a-89cc-1abfb4e08003 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:24.415449 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878891be-3bfd-4c2a-89cc-1abfb4e08003 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:24.415723 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878891be-3bfd-4c2a-89cc-1abfb4e08003 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:24.417001 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878891be-3bfd-4c2a-89cc-1abfb4e08003 None admin] RBAC: Adding query filter params (role.id=83efac3b92eb44e783f4924c9d9ee842, scope.project.id=d861cd3fc0a047d6bf06a6250256ee77, user.id=a97426e846c0416387b714de740cd043) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:24.417319 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878891be-3bfd-4c2a-89cc-1abfb4e08003 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:24.418783 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878891be-3bfd-4c2a-89cc-1abfb4e08003 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:24.425891 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 148/294] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:32:24 2022] GET /identity/v3/role_assignments?user.id=a97426e846c0416387b714de740cd043&scope.project.id=d861cd3fc0a047d6bf06a6250256ee77&role.id=83efac3b92eb44e783f4924c9d9ee842 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:24.788535 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec467405-a7ad-4e8c-b3b0-c1e081360c4b None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:24.789058 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec467405-a7ad-4e8c-b3b0-c1e081360c4b None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:24.789058 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec467405-a7ad-4e8c-b3b0-c1e081360c4b None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:24.790616 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 147/295] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:24.796957 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ebeb8-4b8c-4d31-8cf1-96ef17519bee None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:24.797255 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ebeb8-4b8c-4d31-8cf1-96ef17519bee None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:24.797523 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ebeb8-4b8c-4d31-8cf1-96ef17519bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:24.825219 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-457ebeb8-4b8c-4d31-8cf1-96ef17519bee None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:24.826457 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-457ebeb8-4b8c-4d31-8cf1-96ef17519bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:24.842859 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf3bcc-88c3-4e8e-b23a-fb3805b25be3 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:24.843223 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf3bcc-88c3-4e8e-b23a-fb3805b25be3 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:24.843430 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf3bcc-88c3-4e8e-b23a-fb3805b25be3 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:24.845064 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 148/296] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:32:24.855490 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe25dd9-be6d-4561-8662-be338e366218 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:24.856129 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe25dd9-be6d-4561-8662-be338e366218 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:24.856692 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe25dd9-be6d-4561-8662-be338e366218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:24.885641 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-6fe25dd9-be6d-4561-8662-be338e366218 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:24.886927 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-6fe25dd9-be6d-4561-8662-be338e366218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:24.903576 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:24.927159 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 149/297] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:32:24.934246 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:25.005206 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60630a16-e4c5-4e70-8bb9-0fd12ccf5398 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:25.005740 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60630a16-e4c5-4e70-8bb9-0fd12ccf5398 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:25.006153 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60630a16-e4c5-4e70-8bb9-0fd12ccf5398 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:25.008577 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 150/298] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:25.026750 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 149/299] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 02:32:25.028720 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be33da6-4b01-4d06-bb37-ae9ace1d737e None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:25.029141 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be33da6-4b01-4d06-bb37-ae9ace1d737e None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:25.029551 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be33da6-4b01-4d06-bb37-ae9ace1d737e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:25.067616 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-5be33da6-4b01-4d06-bb37-ae9ace1d737e None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:25.069691 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-5be33da6-4b01-4d06-bb37-ae9ace1d737e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:25.206127 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 151/300] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:32:25.319516 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe25dd9-be6d-4561-8662-be338e366218 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:25.412709 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be33da6-4b01-4d06-bb37-ae9ace1d737e None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:25.469577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe25dd9-be6d-4561-8662-be338e366218 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:25.546075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4fccc-4fd0-4f38-b69e-dae9fe526b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:25.547632 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4fccc-4fd0-4f38-b69e-dae9fe526b96 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:25.548141 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4fccc-4fd0-4f38-b69e-dae9fe526b96 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:25.548889 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4fccc-4fd0-4f38-b69e-dae9fe526b96 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:25.563585 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be33da6-4b01-4d06-bb37-ae9ace1d737e None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:25.567162 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4fccc-4fd0-4f38-b69e-dae9fe526b96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:25.573835 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-cab4fccc-4fd0-4f38-b69e-dae9fe526b96 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 02:32:25.573835 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:25.573835 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:25.573835 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:25.573835 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:25.573835 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:25.573835 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:25.573835 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:25.573835 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:25.573835 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:25.573835 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:25.573835 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:25.573835 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:25.573835 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:25.573835 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:25.573835 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:25.574760 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:25.574760 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:25.574760 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:25.574760 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:25.574760 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:25.574760 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:25.574760 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:25.574760 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:25.574760 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:25.574760 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:25.574760 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:25.574760 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:25.574760 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:25.574760 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:25.574760 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:25.574760 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:25.575625 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:25.575625 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:25.575625 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:25.575625 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:25.575625 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:25.575625 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:25.575625 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:25.575625 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:25.575625 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:25.575625 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:25.575625 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 02:32:25.575625 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:32:25.579195 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 150/301] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:32:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:25.591049 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4fccc-4fd0-4f38-b69e-dae9fe526b96 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:25.594259 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:25.626387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68910451-eb35-4401-8219-5c6982c4644c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:25.627549 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68910451-eb35-4401-8219-5c6982c4644c None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:25.629047 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68910451-eb35-4401-8219-5c6982c4644c None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:25.631472 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68910451-eb35-4401-8219-5c6982c4644c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:25.640145 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:25.659561 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68910451-eb35-4401-8219-5c6982c4644c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:25.673335 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-68910451-eb35-4401-8219-5c6982c4644c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 02:32:25.673335 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:25.673335 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:25.673335 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:25.673335 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:25.673335 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:25.673335 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:25.673335 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:25.673335 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:25.673335 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:25.673335 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:25.673335 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:25.673335 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:25.673335 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:25.673335 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:25.673335 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:25.674304 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:25.674304 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:25.674304 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:25.674304 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:25.674304 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:25.674304 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:25.674304 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:25.674304 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:25.674304 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:25.674304 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:25.674304 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:25.674304 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:25.674304 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:25.674304 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:25.674304 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:25.674304 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:25.675185 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:25.675185 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:25.675185 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:25.675185 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:25.675185 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:25.675185 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:25.675185 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:25.675185 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:25.675185 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:25.675185 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:25.675185 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 02:32:25.675185 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:32:25.678050 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 152/302] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:32:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:25.685486 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68910451-eb35-4401-8219-5c6982c4644c None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:25.770144 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4fccc-4fd0-4f38-b69e-dae9fe526b96 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:25.820651 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68910451-eb35-4401-8219-5c6982c4644c None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:25.842503 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae13764-875a-41b8-bba0-2718952d4eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:25.843507 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae13764-875a-41b8-bba0-2718952d4eda None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:25.843872 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae13764-875a-41b8-bba0-2718952d4eda None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:25.844349 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae13764-875a-41b8-bba0-2718952d4eda None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:25.845658 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae13764-875a-41b8-bba0-2718952d4eda None admin] RBAC: Adding query filter params (name=member) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:25.846213 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae13764-875a-41b8-bba0-2718952d4eda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:25.849204 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae13764-875a-41b8-bba0-2718952d4eda None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:25.871169 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 151/303] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Sep 22 02:32:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:32:25.872278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae13764-875a-41b8-bba0-2718952d4eda None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:25.880113 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f74f8-dbc1-46f4-8017-5d122dae543e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:25.882590 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f74f8-dbc1-46f4-8017-5d122dae543e None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:25.883421 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f74f8-dbc1-46f4-8017-5d122dae543e None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:25.884110 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f74f8-dbc1-46f4-8017-5d122dae543e None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:25.886069 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472f74f8-dbc1-46f4-8017-5d122dae543e None admin] RBAC: Adding query filter params (name=reader) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:25.886813 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472f74f8-dbc1-46f4-8017-5d122dae543e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:25.889537 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472f74f8-dbc1-46f4-8017-5d122dae543e None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:25.906883 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 153/304] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Sep 22 02:32:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:25.909645 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f74f8-dbc1-46f4-8017-5d122dae543e None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:25.978432 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:26.031356 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:26.047867 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae13764-875a-41b8-bba0-2718952d4eda None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:26.094889 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db55470-8ae4-4ef9-bb48-51bc8e8e3f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:26.095782 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db55470-8ae4-4ef9-bb48-51bc8e8e3f60 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:26.096239 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db55470-8ae4-4ef9-bb48-51bc8e8e3f60 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:26.096652 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db55470-8ae4-4ef9-bb48-51bc8e8e3f60 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:26.098173 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db55470-8ae4-4ef9-bb48-51bc8e8e3f60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:26.099993 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db55470-8ae4-4ef9-bb48-51bc8e8e3f60 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:26.100734 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f74f8-dbc1-46f4-8017-5d122dae543e None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:26.102051 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 152/305] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:32:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:26.108611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db55470-8ae4-4ef9-bb48-51bc8e8e3f60 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:26.165904 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bcf303-8676-479a-a9fc-26ce55582aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:26.166637 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bcf303-8676-479a-a9fc-26ce55582aed None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:26.166951 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bcf303-8676-479a-a9fc-26ce55582aed None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:26.167227 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bcf303-8676-479a-a9fc-26ce55582aed None admin] PATH_INFO: `/v3/users/a26448711c4f4807ac055a00470b80eb` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:26.168478 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bcf303-8676-479a-a9fc-26ce55582aed None admin] RBAC: Authorizing `identity:get_user(user_id=a26448711c4f4807ac055a00470b80eb)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:26.169967 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bcf303-8676-479a-a9fc-26ce55582aed None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:26.179675 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 154/306] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:32:25 2022] GET /identity/v3/users/a26448711c4f4807ac055a00470b80eb => generated 293 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:26.179675 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bcf303-8676-479a-a9fc-26ce55582aed None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:26.284699 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db55470-8ae4-4ef9-bb48-51bc8e8e3f60 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:26.337694 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bcf303-8676-479a-a9fc-26ce55582aed None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:26.345263 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7b3512-0b6c-4d26-a5da-293df4f56e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:26.346625 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b3512-0b6c-4d26-a5da-293df4f56e17 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:26.347365 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b3512-0b6c-4d26-a5da-293df4f56e17 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:26.348087 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b3512-0b6c-4d26-a5da-293df4f56e17 None admin] PATH_INFO: `/v3/users/c72782219aea46e5b7adebcd2c2e91ce` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:26.350663 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:26.376688 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7b3512-0b6c-4d26-a5da-293df4f56e17 None admin] RBAC: Authorizing `identity:get_user(user_id=c72782219aea46e5b7adebcd2c2e91ce)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:26.379226 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7b3512-0b6c-4d26-a5da-293df4f56e17 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:26.381976 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 153/307] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:32:26 2022] GET /identity/v3/users/c72782219aea46e5b7adebcd2c2e91ce => generated 293 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:26.384480 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7b3512-0b6c-4d26-a5da-293df4f56e17 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:26.404749 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e5468-69d6-46a3-b629-ad62ea500370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:26.406240 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e5468-69d6-46a3-b629-ad62ea500370 None admin] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:26.406678 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e5468-69d6-46a3-b629-ad62ea500370 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:26.407133 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e5468-69d6-46a3-b629-ad62ea500370 None admin] PATH_INFO: `/v3/groups` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:26.408105 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48e5468-69d6-46a3-b629-ad62ea500370 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:26.410489 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48e5468-69d6-46a3-b629-ad62ea500370 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:26.416489 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:26.434887 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 155/308] 10.222.0.20 () {60 vars in 1134 bytes} [Thu Sep 22 02:32:26 2022] POST /identity/v3/groups => generated 229 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:32:26.435687 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e5468-69d6-46a3-b629-ad62ea500370 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:26.597267 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7b3512-0b6c-4d26-a5da-293df4f56e17 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:26.623213 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e5468-69d6-46a3-b629-ad62ea500370 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:26.657348 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b91d08-2451-4626-99e5-37e1fd142be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:26.658460 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b91d08-2451-4626-99e5-37e1fd142be0 None admin] REQUEST_METHOD: `PUT` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:26.658973 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b91d08-2451-4626-99e5-37e1fd142be0 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:26.659430 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b91d08-2451-4626-99e5-37e1fd142be0 None admin] PATH_INFO: `/v3/system/users/a26448711c4f4807ac055a00470b80eb/roles/83efac3b92eb44e783f4924c9d9ee842` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:26.661899 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b91d08-2451-4626-99e5-37e1fd142be0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a26448711c4f4807ac055a00470b80eb, role_id=83efac3b92eb44e783f4924c9d9ee842)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:26.664478 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b91d08-2451-4626-99e5-37e1fd142be0 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:26.684093 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 154/309] 10.222.0.20 () {58 vars in 1334 bytes} [Thu Sep 22 02:32:26 2022] PUT /identity/v3/system/users/a26448711c4f4807ac055a00470b80eb/roles/83efac3b92eb44e783f4924c9d9ee842 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 02:32:26.684093 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88063343-cbb8-4373-8de1-7dd6d9d347a8 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:26.684093 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88063343-cbb8-4373-8de1-7dd6d9d347a8 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:26.684093 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88063343-cbb8-4373-8de1-7dd6d9d347a8 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:26.684093 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 155/310] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:26.691454 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0029e-38f4-4a01-bbd5-161a623da508 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:26.691841 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0029e-38f4-4a01-bbd5-161a623da508 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:26.692312 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0029e-38f4-4a01-bbd5-161a623da508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:26.696872 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531400d-1fd3-4260-b64c-c5744b89ee98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:26.698493 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531400d-1fd3-4260-b64c-c5744b89ee98 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:26.699226 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531400d-1fd3-4260-b64c-c5744b89ee98 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:26.699895 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531400d-1fd3-4260-b64c-c5744b89ee98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:26.701608 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c531400d-1fd3-4260-b64c-c5744b89ee98 None admin] RBAC: Adding query filter params (role.id=13d7b9e519274c8bb245c8218571dee8, user.id=c72782219aea46e5b7adebcd2c2e91ce, scope.system=all) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:26.702427 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c531400d-1fd3-4260-b64c-c5744b89ee98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:26.704948 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c531400d-1fd3-4260-b64c-c5744b89ee98 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:26.716333 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 156/311] 10.222.0.20 () {56 vars in 1308 bytes} [Thu Sep 22 02:32:26 2022] GET /identity/v3/role_assignments?user.id=c72782219aea46e5b7adebcd2c2e91ce&scope.system=all&role.id=13d7b9e519274c8bb245c8218571dee8 => generated 226 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:26.725171 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-05a0029e-38f4-4a01-bbd5-161a623da508 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:26.727231 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-05a0029e-38f4-4a01-bbd5-161a623da508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:26.875103 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 156/312] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 02:32:27.376484 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531400d-1fd3-4260-b64c-c5744b89ee98 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:27.440654 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:27.581474 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531400d-1fd3-4260-b64c-c5744b89ee98 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:27.645271 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303fe89d-4131-40e5-8a4f-abe1cce22a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:27.646248 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303fe89d-4131-40e5-8a4f-abe1cce22a68 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:27.646746 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303fe89d-4131-40e5-8a4f-abe1cce22a68 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:27.647274 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303fe89d-4131-40e5-8a4f-abe1cce22a68 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:27.668771 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303fe89d-4131-40e5-8a4f-abe1cce22a68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:27.679875 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-303fe89d-4131-40e5-8a4f-abe1cce22a68 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 02:32:27.679875 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:27.679875 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:27.679875 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:27.679875 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:27.679875 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:27.679875 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:27.679875 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:27.679875 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:27.679875 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:27.679875 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:27.679875 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:27.679875 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:27.679875 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:27.679875 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:27.679875 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:27.681704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:27.681704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:27.681704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:27.681704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:27.681704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:27.681704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:27.681704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:27.681704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:27.681704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:27.681704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:27.681704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:27.681704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:27.681704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:27.681704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:27.681704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:27.681704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:27.683966 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:27.683966 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:27.683966 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:27.683966 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:27.683966 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:27.683966 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:27.683966 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:27.683966 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:27.683966 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:27.683966 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:27.683966 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 02:32:27.683966 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:32:27.683966 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 157/313] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:32:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:27.691752 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a0029e-38f4-4a01-bbd5-161a623da508 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:27.712558 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:27.896669 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a0029e-38f4-4a01-bbd5-161a623da508 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:27.965450 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29229ae-f2b8-4eab-874d-144761589a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:27.966563 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29229ae-f2b8-4eab-874d-144761589a64 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:27.967088 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29229ae-f2b8-4eab-874d-144761589a64 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:27.967570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29229ae-f2b8-4eab-874d-144761589a64 None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:27.968814 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29229ae-f2b8-4eab-874d-144761589a64 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:27.969355 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29229ae-f2b8-4eab-874d-144761589a64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:27.972010 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29229ae-f2b8-4eab-874d-144761589a64 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:27.986345 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 157/314] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Sep 22 02:32:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:27.996720 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303fe89d-4131-40e5-8a4f-abe1cce22a68 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:28.183526 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:28.203039 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303fe89d-4131-40e5-8a4f-abe1cce22a68 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:28.270623 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49be9c6-f7da-4372-954d-999245881269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:28.271782 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49be9c6-f7da-4372-954d-999245881269 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:28.272286 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49be9c6-f7da-4372-954d-999245881269 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:28.272697 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49be9c6-f7da-4372-954d-999245881269 None admin] PATH_INFO: `/v3/users/33130aa187974218bf1b30179a4371e1` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:28.275189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49be9c6-f7da-4372-954d-999245881269 None admin] RBAC: Authorizing `identity:get_user(user_id=33130aa187974218bf1b30179a4371e1)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:28.279054 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49be9c6-f7da-4372-954d-999245881269 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:28.286222 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 158/315] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:32:27 2022] GET /identity/v3/users/33130aa187974218bf1b30179a4371e1 => generated 297 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:28.291916 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29229ae-f2b8-4eab-874d-144761589a64 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:28.430872 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:28.494608 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29229ae-f2b8-4eab-874d-144761589a64 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:28.562680 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b351ac5b-9d2e-4105-8b38-e4fe0be5a0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:28.563770 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b351ac5b-9d2e-4105-8b38-e4fe0be5a0d2 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:28.564253 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b351ac5b-9d2e-4105-8b38-e4fe0be5a0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:28.564744 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b351ac5b-9d2e-4105-8b38-e4fe0be5a0d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:28.688063 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b351ac5b-9d2e-4105-8b38-e4fe0be5a0d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:28.690073 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b351ac5b-9d2e-4105-8b38-e4fe0be5a0d2 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:28.782778 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:28.848917 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 158/316] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:32:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 22 02:32:28.859214 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49be9c6-f7da-4372-954d-999245881269 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:29.058163 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49be9c6-f7da-4372-954d-999245881269 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:29.098111 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef9807-99b1-4c3c-8cb2-a7781d48dde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:29.098768 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef9807-99b1-4c3c-8cb2-a7781d48dde1 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:29.099089 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef9807-99b1-4c3c-8cb2-a7781d48dde1 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:29.099382 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef9807-99b1-4c3c-8cb2-a7781d48dde1 None admin] PATH_INFO: `/v3/projects/d861cd3fc0a047d6bf06a6250256ee77` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:29.100593 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef9807-99b1-4c3c-8cb2-a7781d48dde1 None admin] RBAC: Authorizing `identity:get_project(project_id=d861cd3fc0a047d6bf06a6250256ee77)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:29.102072 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef9807-99b1-4c3c-8cb2-a7781d48dde1 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:29.104691 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 159/317] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:32:28 2022] GET /identity/v3/projects/d861cd3fc0a047d6bf06a6250256ee77 => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:29.111448 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b351ac5b-9d2e-4105-8b38-e4fe0be5a0d2 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:29.285852 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b351ac5b-9d2e-4105-8b38-e4fe0be5a0d2 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:29.326414 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c34b3-63cf-4ce4-a2c0-269df47d4785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:29.327075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c34b3-63cf-4ce4-a2c0-269df47d4785 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:29.327360 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c34b3-63cf-4ce4-a2c0-269df47d4785 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:29.327634 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c34b3-63cf-4ce4-a2c0-269df47d4785 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:29.328392 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c34b3-63cf-4ce4-a2c0-269df47d4785 None admin] RBAC: Adding query filter params (role.id=13d7b9e519274c8bb245c8218571dee8, scope.project.id=d861cd3fc0a047d6bf06a6250256ee77, user.id=33130aa187974218bf1b30179a4371e1) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:29.328706 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c34b3-63cf-4ce4-a2c0-269df47d4785 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:29.330204 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c34b3-63cf-4ce4-a2c0-269df47d4785 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:29.337424 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 159/318] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:32:29 2022] GET /identity/v3/role_assignments?user.id=33130aa187974218bf1b30179a4371e1&scope.project.id=d861cd3fc0a047d6bf06a6250256ee77&role.id=13d7b9e519274c8bb245c8218571dee8 => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:29.613312 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7941be66-e8eb-4ea3-8a0e-8c273ca86977 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:29.613971 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7941be66-e8eb-4ea3-8a0e-8c273ca86977 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:29.614485 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7941be66-e8eb-4ea3-8a0e-8c273ca86977 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:29.616575 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 160/319] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:29.622785 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87f13f-ebe5-4d29-8577-5620168471e9 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:29.623442 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87f13f-ebe5-4d29-8577-5620168471e9 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:29.623925 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87f13f-ebe5-4d29-8577-5620168471e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:29.646090 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-2a87f13f-ebe5-4d29-8577-5620168471e9 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:29.647356 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-2a87f13f-ebe5-4d29-8577-5620168471e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:29.681953 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:29.744720 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 160/320] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 02:32:29.934783 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41cfce2-0789-45b7-bd24-ea055532b394 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:29.935587 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41cfce2-0789-45b7-bd24-ea055532b394 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:29.935587 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41cfce2-0789-45b7-bd24-ea055532b394 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:29.937009 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 161/321] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:29.943812 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7208c-a8be-4d2b-a380-3967bd0efb28 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:29.944368 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7208c-a8be-4d2b-a380-3967bd0efb28 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:29.944848 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7208c-a8be-4d2b-a380-3967bd0efb28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:29.978392 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-88b7208c-a8be-4d2b-a380-3967bd0efb28 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:29.980521 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-88b7208c-a8be-4d2b-a380-3967bd0efb28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:30.017110 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41cfce2-0789-45b7-bd24-ea055532b394 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:30.054552 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:30.102924 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 161/322] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:32:30.114463 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474343a-f6ba-42f9-a69c-ba28b76e5890 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:30.115117 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474343a-f6ba-42f9-a69c-ba28b76e5890 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:30.115602 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474343a-f6ba-42f9-a69c-ba28b76e5890 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:30.118461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 162/323] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:32:30.124782 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc0b3d-b70c-4cd8-ad52-e34435b3db8e None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:30.125079 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc0b3d-b70c-4cd8-ad52-e34435b3db8e None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:30.125407 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc0b3d-b70c-4cd8-ad52-e34435b3db8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:30.145929 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-1dcc0b3d-b70c-4cd8-ad52-e34435b3db8e None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:30.147204 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-1dcc0b3d-b70c-4cd8-ad52-e34435b3db8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:30.161553 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41cfce2-0789-45b7-bd24-ea055532b394 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:30.208916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8384ad-fc0f-4df8-a925-cae28f85be8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:30.209589 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8384ad-fc0f-4df8-a925-cae28f85be8c None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:30.209888 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8384ad-fc0f-4df8-a925-cae28f85be8c None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:30.210165 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8384ad-fc0f-4df8-a925-cae28f85be8c None admin] PATH_INFO: `/v3/domains/default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:30.211333 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8384ad-fc0f-4df8-a925-cae28f85be8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:30.212828 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8384ad-fc0f-4df8-a925-cae28f85be8c None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:30.214756 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 162/324] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:32:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:32:30.220578 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8384ad-fc0f-4df8-a925-cae28f85be8c None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:30.233203 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 163/325] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 02:32:30.331520 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:30.341551 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8384ad-fc0f-4df8-a925-cae28f85be8c None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:30.381800 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998036cf-86bc-4880-9b9c-37bda2d0d0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:30.382529 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998036cf-86bc-4880-9b9c-37bda2d0d0de None admin] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:30.382838 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998036cf-86bc-4880-9b9c-37bda2d0d0de None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:30.383163 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998036cf-86bc-4880-9b9c-37bda2d0d0de None admin] PATH_INFO: `/v3/groups` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:30.383849 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998036cf-86bc-4880-9b9c-37bda2d0d0de None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:30.386049 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998036cf-86bc-4880-9b9c-37bda2d0d0de None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:30.406925 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 163/326] 10.222.0.20 () {60 vars in 1134 bytes} [Thu Sep 22 02:32:30 2022] POST /identity/v3/groups => generated 226 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:32:30.531999 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc0b3d-b70c-4cd8-ad52-e34435b3db8e None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:30.563985 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:30.596217 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998036cf-86bc-4880-9b9c-37bda2d0d0de None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:30.679645 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc0b3d-b70c-4cd8-ad52-e34435b3db8e None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:30.729988 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a60b0-5e48-45c2-bad6-1879ffaecaa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:30.731271 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a60b0-5e48-45c2-bad6-1879ffaecaa2 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:30.731975 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a60b0-5e48-45c2-bad6-1879ffaecaa2 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:30.732617 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a60b0-5e48-45c2-bad6-1879ffaecaa2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:30.753666 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8a60b0-5e48-45c2-bad6-1879ffaecaa2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:30.765902 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-cc8a60b0-5e48-45c2-bad6-1879ffaecaa2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 02:32:30.765902 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:30.765902 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:30.765902 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:30.765902 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:30.765902 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:30.765902 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:30.765902 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:30.765902 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:30.765902 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:30.765902 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:30.765902 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:30.765902 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:30.765902 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:30.765902 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:30.765902 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:30.767734 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:30.767734 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:30.767734 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:30.767734 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:30.767734 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:30.767734 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:30.767734 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:30.767734 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:30.767734 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:30.767734 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:30.767734 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:30.767734 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:30.767734 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:30.767734 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:30.767734 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:30.767734 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:30.769210 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:30.769210 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:30.769210 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:30.769210 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:30.769210 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:30.769210 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:30.769210 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:30.769210 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:30.769210 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:30.769210 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:30.769210 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 02:32:30.769210 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:32:30.769210 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 164/327] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:32:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:32:30.780726 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a60b0-5e48-45c2-bad6-1879ffaecaa2 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:30.785272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998036cf-86bc-4880-9b9c-37bda2d0d0de None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:30.822696 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58134687-b9ad-4fb6-91d9-9ac65f27fb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:30.823384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58134687-b9ad-4fb6-91d9-9ac65f27fb58 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:30.823662 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58134687-b9ad-4fb6-91d9-9ac65f27fb58 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:30.823936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58134687-b9ad-4fb6-91d9-9ac65f27fb58 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:30.847976 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58134687-b9ad-4fb6-91d9-9ac65f27fb58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:30.859083 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-58134687-b9ad-4fb6-91d9-9ac65f27fb58 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 02:32:30.859083 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:30.859083 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:30.859083 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:30.859083 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:30.859083 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:30.859083 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:30.859083 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:30.859083 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:30.859083 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:30.859083 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:30.859083 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:30.859083 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:30.859083 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:30.859083 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:30.859083 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:30.860035 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:30.860035 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:30.860035 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:30.860035 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:30.860035 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:30.860035 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:30.860035 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:30.860035 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:30.860035 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:30.860035 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:30.860035 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:30.860035 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:30.860035 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:30.860035 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:30.860035 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:30.860035 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:30.860811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:30.860811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:30.860811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:30.860811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:30.860811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:30.860811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:30.860811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:30.860811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:30.860811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:30.860811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:30.860811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 02:32:30.860811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:32:30.866266 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 164/328] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:32:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:32:30.868128 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58134687-b9ad-4fb6-91d9-9ac65f27fb58 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:30.887137 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:30.944242 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:30.968004 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a60b0-5e48-45c2-bad6-1879ffaecaa2 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:31.013844 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120725b4-879c-4c5c-8c80-f8df03d0587d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:31.014477 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120725b4-879c-4c5c-8c80-f8df03d0587d None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:31.014763 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120725b4-879c-4c5c-8c80-f8df03d0587d None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:31.015078 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120725b4-879c-4c5c-8c80-f8df03d0587d None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:31.015776 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120725b4-879c-4c5c-8c80-f8df03d0587d None admin] RBAC: Adding query filter params (name=member) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:31.016081 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120725b4-879c-4c5c-8c80-f8df03d0587d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:31.018162 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120725b4-879c-4c5c-8c80-f8df03d0587d None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:31.039263 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 165/329] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Sep 22 02:32:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:31.045229 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120725b4-879c-4c5c-8c80-f8df03d0587d None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:31.059437 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58134687-b9ad-4fb6-91d9-9ac65f27fb58 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:31.115484 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e93602-06d9-4b7b-ac18-afd71f841ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:31.116659 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e93602-06d9-4b7b-ac18-afd71f841ff3 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:31.117093 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e93602-06d9-4b7b-ac18-afd71f841ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:31.117493 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e93602-06d9-4b7b-ac18-afd71f841ff3 None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:31.118198 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e93602-06d9-4b7b-ac18-afd71f841ff3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:31.118605 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e93602-06d9-4b7b-ac18-afd71f841ff3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:31.121743 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e93602-06d9-4b7b-ac18-afd71f841ff3 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:31.132676 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 165/330] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Sep 22 02:32:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:31.139136 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e93602-06d9-4b7b-ac18-afd71f841ff3 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:31.182094 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120725b4-879c-4c5c-8c80-f8df03d0587d None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:31.239308 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08235e2f-b907-44c2-8644-8431adbc3561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:31.240465 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08235e2f-b907-44c2-8644-8431adbc3561 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:31.241082 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08235e2f-b907-44c2-8644-8431adbc3561 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:31.241574 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08235e2f-b907-44c2-8644-8431adbc3561 None admin] PATH_INFO: `/v3/users/a26448711c4f4807ac055a00470b80eb` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:31.243661 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08235e2f-b907-44c2-8644-8431adbc3561 None admin] RBAC: Authorizing `identity:get_user(user_id=a26448711c4f4807ac055a00470b80eb)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:31.246254 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:31.250607 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08235e2f-b907-44c2-8644-8431adbc3561 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:31.255598 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 166/331] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:32:31 2022] GET /identity/v3/users/a26448711c4f4807ac055a00470b80eb => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:32:31.263180 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08235e2f-b907-44c2-8644-8431adbc3561 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:31.287520 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:31.298613 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e93602-06d9-4b7b-ac18-afd71f841ff3 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:31.386577 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14265b-27d9-4906-9425-2c1b298393fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:31.390181 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14265b-27d9-4906-9425-2c1b298393fa None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:31.390745 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14265b-27d9-4906-9425-2c1b298393fa None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:31.391397 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14265b-27d9-4906-9425-2c1b298393fa None admin] PATH_INFO: `/v3/users/c72782219aea46e5b7adebcd2c2e91ce` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:31.393780 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d14265b-27d9-4906-9425-2c1b298393fa None admin] RBAC: Authorizing `identity:get_user(user_id=c72782219aea46e5b7adebcd2c2e91ce)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:31.396785 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d14265b-27d9-4906-9425-2c1b298393fa None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:31.398722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08235e2f-b907-44c2-8644-8431adbc3561 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:31.400302 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 166/332] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:32:31 2022] GET /identity/v3/users/c72782219aea46e5b7adebcd2c2e91ce => generated 293 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:31.410657 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14265b-27d9-4906-9425-2c1b298393fa None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:31.449177 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ac9fb8-c1d4-4daa-8b22-6cbc505d588a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:31.449846 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac9fb8-c1d4-4daa-8b22-6cbc505d588a None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:31.450176 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac9fb8-c1d4-4daa-8b22-6cbc505d588a None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:31.450479 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac9fb8-c1d4-4daa-8b22-6cbc505d588a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:31.451419 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ac9fb8-c1d4-4daa-8b22-6cbc505d588a None admin] RBAC: Adding query filter params (role.id=83efac3b92eb44e783f4924c9d9ee842, user.id=a26448711c4f4807ac055a00470b80eb, scope.system=all) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:31.451737 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ac9fb8-c1d4-4daa-8b22-6cbc505d588a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:31.453465 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ac9fb8-c1d4-4daa-8b22-6cbc505d588a None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:31.468647 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 167/333] 10.222.0.20 () {56 vars in 1308 bytes} [Thu Sep 22 02:32:31 2022] GET /identity/v3/role_assignments?user.id=a26448711c4f4807ac055a00470b80eb&scope.system=all&role.id=83efac3b92eb44e783f4924c9d9ee842 => generated 510 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:31.615987 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14265b-27d9-4906-9425-2c1b298393fa None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:31.660270 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a5079-198c-48db-91d9-9963bb221717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:31.660982 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a5079-198c-48db-91d9-9963bb221717 None admin] REQUEST_METHOD: `PUT` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:31.661266 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a5079-198c-48db-91d9-9963bb221717 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:31.661549 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a5079-198c-48db-91d9-9963bb221717 None admin] PATH_INFO: `/v3/system/users/c72782219aea46e5b7adebcd2c2e91ce/roles/13d7b9e519274c8bb245c8218571dee8` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:31.663315 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a5079-198c-48db-91d9-9963bb221717 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c72782219aea46e5b7adebcd2c2e91ce, role_id=13d7b9e519274c8bb245c8218571dee8)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:31.664953 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a5079-198c-48db-91d9-9963bb221717 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:31.677456 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 167/334] 10.222.0.20 () {58 vars in 1334 bytes} [Thu Sep 22 02:32:31 2022] PUT /identity/v3/system/users/c72782219aea46e5b7adebcd2c2e91ce/roles/13d7b9e519274c8bb245c8218571dee8 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 02:32:33.759991 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11003d8d-ad31-418c-88a6-8034a7caeccb None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:33.761630 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11003d8d-ad31-418c-88a6-8034a7caeccb None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:33.762374 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11003d8d-ad31-418c-88a6-8034a7caeccb None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:33.765505 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 168/335] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:33.771846 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f2923-818d-4c23-871b-8c78bc9259d8 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:33.773242 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f2923-818d-4c23-871b-8c78bc9259d8 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:33.773953 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f2923-818d-4c23-871b-8c78bc9259d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:33.811781 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-3b9f2923-818d-4c23-871b-8c78bc9259d8 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:33.814123 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-3b9f2923-818d-4c23-871b-8c78bc9259d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:33.819172 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:33.969927 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 168/336] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:32:34.172539 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458a1fc-6d65-46f5-858f-37ecdfb91b7a None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:34.173546 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458a1fc-6d65-46f5-858f-37ecdfb91b7a None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:34.174334 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458a1fc-6d65-46f5-858f-37ecdfb91b7a None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:34.177194 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 169/337] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:34.187002 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ee0fc-5486-44bc-864b-433dc91dd318 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:34.188027 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ee0fc-5486-44bc-864b-433dc91dd318 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:34.188802 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ee0fc-5486-44bc-864b-433dc91dd318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:34.225442 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-bf6ee0fc-5486-44bc-864b-433dc91dd318 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:34.227939 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-bf6ee0fc-5486-44bc-864b-433dc91dd318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:34.380415 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 169/338] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:32:34.529103 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b458a1fc-6d65-46f5-858f-37ecdfb91b7a None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:34.623169 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:34.662077 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b458a1fc-6d65-46f5-858f-37ecdfb91b7a None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:34.704241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983417f8-2ed8-46dd-b8c0-b18943309d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:34.704942 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983417f8-2ed8-46dd-b8c0-b18943309d1f None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:34.705232 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983417f8-2ed8-46dd-b8c0-b18943309d1f None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:34.705505 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983417f8-2ed8-46dd-b8c0-b18943309d1f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:34.724347 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983417f8-2ed8-46dd-b8c0-b18943309d1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:34.730598 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-983417f8-2ed8-46dd-b8c0-b18943309d1f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 02:32:34.730598 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:34.730598 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:34.730598 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:34.730598 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:34.730598 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:34.730598 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:34.730598 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:34.730598 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:34.730598 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:34.730598 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:34.730598 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:34.730598 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:34.730598 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:34.730598 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:34.730598 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:34.731592 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:34.731592 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:34.731592 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:34.731592 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:34.731592 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:34.731592 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:34.731592 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:34.731592 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:34.731592 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:34.731592 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:34.731592 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:34.731592 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:34.731592 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:34.731592 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:34.731592 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:34.731592 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:34.732380 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:34.732380 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:34.732380 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:34.732380 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:34.732380 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:34.732380 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:34.732380 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:34.732380 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:34.732380 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:34.732380 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:34.732380 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 02:32:34.732380 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:32:34.732380 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 170/339] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Sep 22 02:32:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:34.738154 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ee0fc-5486-44bc-864b-433dc91dd318 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:34.861257 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:34.909735 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ee0fc-5486-44bc-864b-433dc91dd318 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:34.958382 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d219688-72fb-4748-87c0-bda71e26c7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:34.959397 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d219688-72fb-4748-87c0-bda71e26c7f4 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:34.959885 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d219688-72fb-4748-87c0-bda71e26c7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:34.960356 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d219688-72fb-4748-87c0-bda71e26c7f4 None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:34.961255 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d219688-72fb-4748-87c0-bda71e26c7f4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:34.961799 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d219688-72fb-4748-87c0-bda71e26c7f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:34.964058 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d219688-72fb-4748-87c0-bda71e26c7f4 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:34.978408 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 170/340] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:32:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:32:34.984441 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983417f8-2ed8-46dd-b8c0-b18943309d1f None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:35.155061 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983417f8-2ed8-46dd-b8c0-b18943309d1f None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:35.207350 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:35.209270 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510116fb-a8cb-4cc3-bd04-646e9651a04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:35.209893 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510116fb-a8cb-4cc3-bd04-646e9651a04f None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:35.210496 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510116fb-a8cb-4cc3-bd04-646e9651a04f None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:35.210813 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510116fb-a8cb-4cc3-bd04-646e9651a04f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:35.303696 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510116fb-a8cb-4cc3-bd04-646e9651a04f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:35.305226 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510116fb-a8cb-4cc3-bd04-646e9651a04f None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:35.356878 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d219688-72fb-4748-87c0-bda71e26c7f4 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:35.473804 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 171/341] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:32:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 02:32:35.479730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc940cb-5443-4bd9-9af3-6a23cf9b95a8 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:35.480083 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc940cb-5443-4bd9-9af3-6a23cf9b95a8 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:35.480303 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc940cb-5443-4bd9-9af3-6a23cf9b95a8 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:35.481809 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 172/342] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:35.485000 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc940cb-5443-4bd9-9af3-6a23cf9b95a8 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:35.542238 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:35.568527 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d219688-72fb-4748-87c0-bda71e26c7f4 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:35.616661 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:35.629551 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab87ad9-c23a-4226-a000-1063f51861e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:35.630342 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab87ad9-c23a-4226-a000-1063f51861e8 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:35.630664 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab87ad9-c23a-4226-a000-1063f51861e8 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:35.630975 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab87ad9-c23a-4226-a000-1063f51861e8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:35.652147 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc940cb-5443-4bd9-9af3-6a23cf9b95a8 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:35.652857 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab87ad9-c23a-4226-a000-1063f51861e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:35.658926 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-2ab87ad9-c23a-4226-a000-1063f51861e8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 02:32:35.658926 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:35.658926 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:35.658926 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:35.658926 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:35.658926 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:35.658926 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:35.658926 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:35.658926 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:35.658926 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:35.658926 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:35.658926 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:35.658926 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:35.658926 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:35.658926 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:35.658926 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:35.659882 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:35.659882 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:35.659882 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:35.659882 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:35.659882 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:35.659882 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:35.659882 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:35.659882 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:35.659882 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:35.659882 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:35.659882 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:35.659882 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:35.659882 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:35.659882 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:35.659882 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:35.659882 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:35.660682 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:35.660682 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:35.660682 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:35.660682 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:35.660682 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:35.660682 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:35.660682 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:35.660682 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:35.660682 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:35.660682 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:35.660682 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 02:32:35.660682 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:32:35.660682 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 171/343] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:32:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:35.661834 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f11708-51ae-4379-a15e-673c9d70aa1f None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:35.661834 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f11708-51ae-4379-a15e-673c9d70aa1f None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:35.661986 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f11708-51ae-4379-a15e-673c9d70aa1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:35.697239 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-31f11708-51ae-4379-a15e-673c9d70aa1f None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:35.698465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-31f11708-51ae-4379-a15e-673c9d70aa1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:35.701919 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2720a7a-7cf9-420e-8e77-012a5e720573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:35.702581 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2720a7a-7cf9-420e-8e77-012a5e720573 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:35.702889 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2720a7a-7cf9-420e-8e77-012a5e720573 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:35.703166 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2720a7a-7cf9-420e-8e77-012a5e720573 None admin] PATH_INFO: `/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:35.704363 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2720a7a-7cf9-420e-8e77-012a5e720573 None admin] RBAC: Authorizing `identity:get_project(project_id=6a5cfaff6dc7495f9a0648c76c9f6eb7)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:35.705829 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2720a7a-7cf9-420e-8e77-012a5e720573 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:35.708107 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 173/344] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:32:35 2022] GET /identity/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:35.708963 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2720a7a-7cf9-420e-8e77-012a5e720573 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:35.808837 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 172/345] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:32:35.810701 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c919b155-950d-4050-8e9d-fa0673ffdf24 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:35.811060 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c919b155-950d-4050-8e9d-fa0673ffdf24 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:35.811430 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c919b155-950d-4050-8e9d-fa0673ffdf24 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:35.814878 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 173/346] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:35.814878 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f47354-81b6-4574-8031-6b4b8c870080 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:35.814878 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f47354-81b6-4574-8031-6b4b8c870080 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:35.814878 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f47354-81b6-4574-8031-6b4b8c870080 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:35.818852 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 174/347] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:35.818852 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f47354-81b6-4574-8031-6b4b8c870080 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:35.884554 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:35.891393 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2720a7a-7cf9-420e-8e77-012a5e720573 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:35.969620 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839434c1-0305-4558-bee8-cace72710ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:35.970246 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839434c1-0305-4558-bee8-cace72710ca6 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:35.970531 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839434c1-0305-4558-bee8-cace72710ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:35.970870 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839434c1-0305-4558-bee8-cace72710ca6 None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:35.971552 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839434c1-0305-4558-bee8-cace72710ca6 None admin] RBAC: Adding query filter params (name=member) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:35.971859 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839434c1-0305-4558-bee8-cace72710ca6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:35.973261 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839434c1-0305-4558-bee8-cace72710ca6 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:35.982490 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f47354-81b6-4574-8031-6b4b8c870080 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:35.986906 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 174/348] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Sep 22 02:32:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:35.986906 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549c6bd-1759-439b-8fea-9dac186b0c47 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:35.986906 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549c6bd-1759-439b-8fea-9dac186b0c47 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:35.986906 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549c6bd-1759-439b-8fea-9dac186b0c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:36.020610 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:36.033405 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-2549c6bd-1759-439b-8fea-9dac186b0c47 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:36.036554 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-2549c6bd-1759-439b-8fea-9dac186b0c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:36.045613 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d500398-ca3d-4318-b8ea-53eb4330f880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:36.046320 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d500398-ca3d-4318-b8ea-53eb4330f880 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:36.046607 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d500398-ca3d-4318-b8ea-53eb4330f880 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:36.047961 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d500398-ca3d-4318-b8ea-53eb4330f880 None admin] PATH_INFO: `/v3/groups/2c48871136ff46089e386a696762e416` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:36.056444 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d500398-ca3d-4318-b8ea-53eb4330f880 None admin] RBAC: Authorizing `identity:get_group(group_id=2c48871136ff46089e386a696762e416)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:36.058640 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d500398-ca3d-4318-b8ea-53eb4330f880 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:36.066267 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 175/349] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Sep 22 02:32:35 2022] GET /identity/v3/groups/2c48871136ff46089e386a696762e416 => generated 226 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:36.070674 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0c727-5d21-4e14-99b0-821e929cd093 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:36.071270 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0c727-5d21-4e14-99b0-821e929cd093 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:36.074849 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0c727-5d21-4e14-99b0-821e929cd093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:36.104978 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-bcb0c727-5d21-4e14-99b0-821e929cd093 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:36.107066 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-bcb0c727-5d21-4e14-99b0-821e929cd093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:36.141697 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 175/350] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:32:36.142891 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549c6bd-1759-439b-8fea-9dac186b0c47 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:36.251072 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 176/351] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:32:36.252635 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb0c727-5d21-4e14-99b0-821e929cd093 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:36.266029 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:36.277552 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549c6bd-1759-439b-8fea-9dac186b0c47 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:36.372335 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e789f5-e829-4271-8d39-66a8466decbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:36.373604 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e789f5-e829-4271-8d39-66a8466decbb None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:36.374097 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e789f5-e829-4271-8d39-66a8466decbb None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:36.374585 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e789f5-e829-4271-8d39-66a8466decbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:36.420457 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb0c727-5d21-4e14-99b0-821e929cd093 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:36.442105 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:36.466349 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e9b52-0c1a-4741-9943-c56f81b700e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:36.467164 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e9b52-0c1a-4741-9943-c56f81b700e0 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:36.467446 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e9b52-0c1a-4741-9943-c56f81b700e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:36.467717 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e9b52-0c1a-4741-9943-c56f81b700e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:36.468460 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1e9b52-0c1a-4741-9943-c56f81b700e0 None admin] RBAC: Adding query filter params (group.id=2c48871136ff46089e386a696762e416, role.id=adec3004b56243fa923bd9f0082a2728, scope.project.id=6a5cfaff6dc7495f9a0648c76c9f6eb7) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:36.468772 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1e9b52-0c1a-4741-9943-c56f81b700e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:36.470595 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1e9b52-0c1a-4741-9943-c56f81b700e0 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:36.477297 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 177/352] 10.222.0.20 () {56 vars in 1376 bytes} [Thu Sep 22 02:32:36 2022] GET /identity/v3/role_assignments?group.id=2c48871136ff46089e386a696762e416&scope.project.id=6a5cfaff6dc7495f9a0648c76c9f6eb7&role.id=adec3004b56243fa923bd9f0082a2728 => generated 260 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:36.478865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e9b52-0c1a-4741-9943-c56f81b700e0 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:36.562297 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e789f5-e829-4271-8d39-66a8466decbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:36.565341 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e789f5-e829-4271-8d39-66a8466decbb None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:36.604942 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e9b52-0c1a-4741-9943-c56f81b700e0 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:36.647071 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cbc1ad-b2d8-4d4b-afea-70382607f899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:36.647654 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbc1ad-b2d8-4d4b-afea-70382607f899 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:36.647933 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbc1ad-b2d8-4d4b-afea-70382607f899 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:36.648205 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbc1ad-b2d8-4d4b-afea-70382607f899 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:36.661932 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cbc1ad-b2d8-4d4b-afea-70382607f899 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:36.668649 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-d0cbc1ad-b2d8-4d4b-afea-70382607f899 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 02:32:36.668649 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:36.668649 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:36.668649 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:36.668649 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:36.668649 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:36.668649 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:36.668649 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:36.668649 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:36.668649 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:36.668649 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:36.668649 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:36.668649 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:36.668649 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:36.668649 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:36.668649 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:36.670878 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:36.670878 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:36.670878 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:36.670878 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:36.670878 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:36.670878 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:36.670878 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:36.670878 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:36.670878 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:36.670878 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:36.670878 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:36.670878 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:36.670878 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:36.670878 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:36.670878 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:36.670878 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:36.672867 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:36.672867 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:36.672867 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:36.672867 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:36.672867 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:36.672867 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:36.672867 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:36.672867 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:36.672867 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:36.672867 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:36.672867 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 02:32:36.672867 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:32:36.672867 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 178/353] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Sep 22 02:32:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:36.672867 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cbc1ad-b2d8-4d4b-afea-70382607f899 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:36.733202 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:36.807265 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:36.819408 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 176/354] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:32:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 677 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 02:32:36.820104 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e789f5-e829-4271-8d39-66a8466decbb None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:36.841906 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cbc1ad-b2d8-4d4b-afea-70382607f899 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:36.891285 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f8dc4e-ed0e-438d-905f-7a43f059cbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:36.892409 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f8dc4e-ed0e-438d-905f-7a43f059cbc0 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:36.892869 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f8dc4e-ed0e-438d-905f-7a43f059cbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:36.893152 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f8dc4e-ed0e-438d-905f-7a43f059cbc0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:36.908421 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f8dc4e-ed0e-438d-905f-7a43f059cbc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:36.913575 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-e9f8dc4e-ed0e-438d-905f-7a43f059cbc0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 02:32:36.913575 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:36.913575 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:36.913575 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:36.913575 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:36.913575 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:36.913575 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:36.913575 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:36.913575 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:36.913575 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:36.913575 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:36.913575 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:36.913575 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:36.913575 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:36.913575 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:36.913575 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:36.914542 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:36.914542 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:36.914542 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:36.914542 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:36.914542 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:36.914542 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:36.914542 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:36.914542 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:36.914542 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:36.914542 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:36.914542 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:36.914542 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:36.914542 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:36.914542 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:36.914542 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:36.914542 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:36.915991 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:36.915991 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:36.915991 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:36.915991 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:36.915991 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:36.915991 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:36.915991 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:36.915991 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:36.915991 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:36.915991 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:36.915991 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 02:32:36.915991 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:32:36.915991 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 179/355] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:32:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:36.921925 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f8dc4e-ed0e-438d-905f-7a43f059cbc0 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:36.961438 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e789f5-e829-4271-8d39-66a8466decbb None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:37.020565 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d562e7-b81f-4a15-ba84-5718ca2c65f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:37.021186 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d562e7-b81f-4a15-ba84-5718ca2c65f5 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:37.026850 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d562e7-b81f-4a15-ba84-5718ca2c65f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:37.026850 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d562e7-b81f-4a15-ba84-5718ca2c65f5 None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:37.026850 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d562e7-b81f-4a15-ba84-5718ca2c65f5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:37.026850 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d562e7-b81f-4a15-ba84-5718ca2c65f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:37.026850 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d562e7-b81f-4a15-ba84-5718ca2c65f5 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:37.036300 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 177/356] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:32:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:37.037302 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d562e7-b81f-4a15-ba84-5718ca2c65f5 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:37.076255 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:37.082446 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f8dc4e-ed0e-438d-905f-7a43f059cbc0 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:37.115001 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:37.139280 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff2c351-7c45-416a-8f5e-54dba4d3000a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:37.140327 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2c351-7c45-416a-8f5e-54dba4d3000a None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:37.140802 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2c351-7c45-416a-8f5e-54dba4d3000a None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:37.141276 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2c351-7c45-416a-8f5e-54dba4d3000a None admin] PATH_INFO: `/v3/roles/member` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:37.161699 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff2c351-7c45-416a-8f5e-54dba4d3000a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:37.167301 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-eff2c351-7c45-416a-8f5e-54dba4d3000a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 02:32:37.167301 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:37.167301 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:37.167301 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:37.167301 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:37.167301 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:37.167301 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:37.167301 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:37.167301 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:37.167301 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:37.167301 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:37.167301 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:37.167301 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:37.167301 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:37.167301 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:37.167301 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:37.168346 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:37.168346 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:37.168346 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:37.168346 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:37.168346 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:37.168346 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:37.168346 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:37.168346 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:37.168346 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:37.168346 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:37.168346 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:37.168346 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:37.168346 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:37.168346 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:37.168346 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:37.168346 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:37.169166 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:37.169166 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:37.169166 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:37.169166 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:37.169166 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:37.169166 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:37.169166 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:37.169166 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:37.169166 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:37.169166 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:37.169166 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 02:32:37.169166 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:32:37.172421 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 180/357] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:32:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:37.174457 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18d7e4-5e78-4466-9a2d-230dc9140ac0 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:37.175021 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18d7e4-5e78-4466-9a2d-230dc9140ac0 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:37.175629 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18d7e4-5e78-4466-9a2d-230dc9140ac0 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:37.177167 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 181/358] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:37.178124 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c18d7e4-5e78-4466-9a2d-230dc9140ac0 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:37.205695 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d562e7-b81f-4a15-ba84-5718ca2c65f5 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:37.256155 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394de9b-44b9-4297-b54a-146310be3831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:37.257132 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394de9b-44b9-4297-b54a-146310be3831 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:37.257597 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394de9b-44b9-4297-b54a-146310be3831 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:37.258077 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394de9b-44b9-4297-b54a-146310be3831 None admin] PATH_INFO: `/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:37.260135 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9394de9b-44b9-4297-b54a-146310be3831 None admin] RBAC: Authorizing `identity:get_project(project_id=6a5cfaff6dc7495f9a0648c76c9f6eb7)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:37.261868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9394de9b-44b9-4297-b54a-146310be3831 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:37.263627 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 178/359] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:32:37 2022] GET /identity/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:37.264494 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394de9b-44b9-4297-b54a-146310be3831 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:37.392869 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c18d7e4-5e78-4466-9a2d-230dc9140ac0 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:37.459940 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310653ec-ef76-44f1-a274-3183f9f7475e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:37.461259 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310653ec-ef76-44f1-a274-3183f9f7475e None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:37.461856 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310653ec-ef76-44f1-a274-3183f9f7475e None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:37.463493 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310653ec-ef76-44f1-a274-3183f9f7475e None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:37.464371 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310653ec-ef76-44f1-a274-3183f9f7475e None admin] RBAC: Adding query filter params (name=reader) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:37.464859 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310653ec-ef76-44f1-a274-3183f9f7475e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:37.467612 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310653ec-ef76-44f1-a274-3183f9f7475e None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:37.482601 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 182/360] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Sep 22 02:32:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:37.484612 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310653ec-ef76-44f1-a274-3183f9f7475e None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:37.490375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394de9b-44b9-4297-b54a-146310be3831 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:37.530673 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:37.536159 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:37.565348 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5b08e-a966-41d8-8a08-950a65da87df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:37.566362 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5b08e-a966-41d8-8a08-950a65da87df None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:37.566880 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5b08e-a966-41d8-8a08-950a65da87df None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:37.567388 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5b08e-a966-41d8-8a08-950a65da87df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:37.678662 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd5b08e-a966-41d8-8a08-950a65da87df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:37.680588 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd5b08e-a966-41d8-8a08-950a65da87df None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:37.701499 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310653ec-ef76-44f1-a274-3183f9f7475e None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:37.767182 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e53328a-2274-48a0-bb36-befe3290048e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:37.768436 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53328a-2274-48a0-bb36-befe3290048e None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:37.769803 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53328a-2274-48a0-bb36-befe3290048e None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:37.769935 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53328a-2274-48a0-bb36-befe3290048e None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:37.771264 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e53328a-2274-48a0-bb36-befe3290048e None admin] RBAC: Adding query filter params (name=member) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:37.771826 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e53328a-2274-48a0-bb36-befe3290048e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:37.775250 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e53328a-2274-48a0-bb36-befe3290048e None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:37.795381 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 183/361] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Sep 22 02:32:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:37.797834 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70879ffc-0409-4b55-8dd7-807a73cf05ba None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:37.798156 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70879ffc-0409-4b55-8dd7-807a73cf05ba None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:37.798444 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70879ffc-0409-4b55-8dd7-807a73cf05ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:37.820343 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-70879ffc-0409-4b55-8dd7-807a73cf05ba None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:37.821623 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-70879ffc-0409-4b55-8dd7-807a73cf05ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:37.846251 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 179/362] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:32:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 02:32:37.846251 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5b08e-a966-41d8-8a08-950a65da87df None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:37.883278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:37.925930 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 184/363] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:32:37.926810 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70879ffc-0409-4b55-8dd7-807a73cf05ba None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:37.942809 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:38.002639 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5b08e-a966-41d8-8a08-950a65da87df None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:38.066865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70879ffc-0409-4b55-8dd7-807a73cf05ba None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:38.075561 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52c755-d4ad-4559-bebc-b37923f35b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:38.075561 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52c755-d4ad-4559-bebc-b37923f35b86 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:38.075561 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52c755-d4ad-4559-bebc-b37923f35b86 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:38.075561 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52c755-d4ad-4559-bebc-b37923f35b86 None admin] PATH_INFO: `/v3/groups/2c48871136ff46089e386a696762e416` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:38.077238 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd52c755-d4ad-4559-bebc-b37923f35b86 None admin] RBAC: Authorizing `identity:get_group(group_id=2c48871136ff46089e386a696762e416)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:38.079143 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd52c755-d4ad-4559-bebc-b37923f35b86 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:38.082429 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 180/364] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Sep 22 02:32:37 2022] GET /identity/v3/groups/2c48871136ff46089e386a696762e416 => generated 226 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:38.083796 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52c755-d4ad-4559-bebc-b37923f35b86 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:38.134612 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85500b72-30da-474c-9475-e6c30965e586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:38.135417 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85500b72-30da-474c-9475-e6c30965e586 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:38.135717 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85500b72-30da-474c-9475-e6c30965e586 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:38.136003 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85500b72-30da-474c-9475-e6c30965e586 None admin] PATH_INFO: `/v3/users/c72782219aea46e5b7adebcd2c2e91ce` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:38.137667 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85500b72-30da-474c-9475-e6c30965e586 None admin] RBAC: Authorizing `identity:get_user(user_id=c72782219aea46e5b7adebcd2c2e91ce)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:38.139330 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85500b72-30da-474c-9475-e6c30965e586 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:38.142294 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 185/365] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:32:37 2022] GET /identity/v3/users/c72782219aea46e5b7adebcd2c2e91ce => generated 293 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:38.143301 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85500b72-30da-474c-9475-e6c30965e586 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:38.229824 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:38.244115 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52c755-d4ad-4559-bebc-b37923f35b86 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:38.267433 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:38.306439 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85500b72-30da-474c-9475-e6c30965e586 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:38.313367 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17405251-a980-4dab-b552-5405ea4e7c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:38.314451 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17405251-a980-4dab-b552-5405ea4e7c4b None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:38.315010 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17405251-a980-4dab-b552-5405ea4e7c4b None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:38.315501 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17405251-a980-4dab-b552-5405ea4e7c4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:38.379043 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf03c01-0709-4c54-b26c-845b39a69b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:38.379594 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf03c01-0709-4c54-b26c-845b39a69b3a None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:38.380728 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf03c01-0709-4c54-b26c-845b39a69b3a None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:38.381008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf03c01-0709-4c54-b26c-845b39a69b3a None admin] PATH_INFO: `/v3/projects/d861cd3fc0a047d6bf06a6250256ee77` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:38.382256 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf03c01-0709-4c54-b26c-845b39a69b3a None admin] RBAC: Authorizing `identity:get_project(project_id=d861cd3fc0a047d6bf06a6250256ee77)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:38.383954 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf03c01-0709-4c54-b26c-845b39a69b3a None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:38.386138 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 186/366] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:32:38 2022] GET /identity/v3/projects/d861cd3fc0a047d6bf06a6250256ee77 => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:38.387059 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf03c01-0709-4c54-b26c-845b39a69b3a None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:38.407634 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17405251-a980-4dab-b552-5405ea4e7c4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:38.409637 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17405251-a980-4dab-b552-5405ea4e7c4b None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:38.523389 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf03c01-0709-4c54-b26c-845b39a69b3a None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:38.559195 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:38.574015 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd07b8e7-e91b-4df9-9db5-09bdc26533f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:38.574297 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 181/367] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:32:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 02:32:38.575146 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd07b8e7-e91b-4df9-9db5-09bdc26533f0 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:38.575305 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17405251-a980-4dab-b552-5405ea4e7c4b None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:38.575741 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd07b8e7-e91b-4df9-9db5-09bdc26533f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:38.576221 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd07b8e7-e91b-4df9-9db5-09bdc26533f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:38.577159 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd07b8e7-e91b-4df9-9db5-09bdc26533f0 None admin] RBAC: Adding query filter params (group.id=2c48871136ff46089e386a696762e416, role.id=83efac3b92eb44e783f4924c9d9ee842, scope.project.id=6a5cfaff6dc7495f9a0648c76c9f6eb7) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:38.577654 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd07b8e7-e91b-4df9-9db5-09bdc26533f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:38.580618 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd07b8e7-e91b-4df9-9db5-09bdc26533f0 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:38.588341 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 187/368] 10.222.0.20 () {56 vars in 1376 bytes} [Thu Sep 22 02:32:38 2022] GET /identity/v3/role_assignments?group.id=2c48871136ff46089e386a696762e416&scope.project.id=6a5cfaff6dc7495f9a0648c76c9f6eb7&role.id=83efac3b92eb44e783f4924c9d9ee842 => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:38.589581 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd07b8e7-e91b-4df9-9db5-09bdc26533f0 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:38.621172 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:38.735787 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17405251-a980-4dab-b552-5405ea4e7c4b None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:38.736368 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd07b8e7-e91b-4df9-9db5-09bdc26533f0 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:38.797976 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795eb40-4bb9-48ad-9254-84925fe079ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:38.798614 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795eb40-4bb9-48ad-9254-84925fe079ad None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:38.799084 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795eb40-4bb9-48ad-9254-84925fe079ad None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:38.799367 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795eb40-4bb9-48ad-9254-84925fe079ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:38.800251 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b795eb40-4bb9-48ad-9254-84925fe079ad None admin] RBAC: Adding query filter params (role.id=13d7b9e519274c8bb245c8218571dee8, user.id=c72782219aea46e5b7adebcd2c2e91ce, scope.system=all) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:38.800854 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b795eb40-4bb9-48ad-9254-84925fe079ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:38.802079 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b795eb40-4bb9-48ad-9254-84925fe079ad None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:38.811052 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77093cd-a088-40fb-93f6-68332bae4458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:38.811803 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77093cd-a088-40fb-93f6-68332bae4458 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:38.812307 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77093cd-a088-40fb-93f6-68332bae4458 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:38.812724 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77093cd-a088-40fb-93f6-68332bae4458 None admin] PATH_INFO: `/v3/groups/2c48871136ff46089e386a696762e416` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:38.814129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77093cd-a088-40fb-93f6-68332bae4458 None admin] RBAC: Authorizing `identity:get_group(group_id=2c48871136ff46089e386a696762e416)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:38.815871 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77093cd-a088-40fb-93f6-68332bae4458 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:38.818188 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 188/369] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Sep 22 02:32:38 2022] GET /identity/v3/groups/2c48871136ff46089e386a696762e416 => generated 226 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:38.819137 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77093cd-a088-40fb-93f6-68332bae4458 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:38.821279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 182/370] 10.222.0.20 () {56 vars in 1308 bytes} [Thu Sep 22 02:32:38 2022] GET /identity/v3/role_assignments?user.id=c72782219aea46e5b7adebcd2c2e91ce&scope.system=all&role.id=13d7b9e519274c8bb245c8218571dee8 => generated 510 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:38.823149 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795eb40-4bb9-48ad-9254-84925fe079ad None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:38.929584 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:39.049233 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:39.051044 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795eb40-4bb9-48ad-9254-84925fe079ad None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:39.057066 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77093cd-a088-40fb-93f6-68332bae4458 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:39.116883 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b416d4-dab8-4582-9d99-a3f7a41bfde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:39.117702 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b416d4-dab8-4582-9d99-a3f7a41bfde3 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:39.117873 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b416d4-dab8-4582-9d99-a3f7a41bfde3 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:39.118248 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b416d4-dab8-4582-9d99-a3f7a41bfde3 None admin] PATH_INFO: `/v3/projects/d861cd3fc0a047d6bf06a6250256ee77` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:39.120528 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b416d4-dab8-4582-9d99-a3f7a41bfde3 None admin] RBAC: Authorizing `identity:get_project(project_id=d861cd3fc0a047d6bf06a6250256ee77)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:39.122075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b416d4-dab8-4582-9d99-a3f7a41bfde3 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:39.125667 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 183/371] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:32:38 2022] GET /identity/v3/projects/d861cd3fc0a047d6bf06a6250256ee77 => generated 302 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:39.126899 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e207a-a638-4308-92c3-d92bd56df175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:39.127576 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e207a-a638-4308-92c3-d92bd56df175 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:39.127870 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e207a-a638-4308-92c3-d92bd56df175 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:39.128136 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e207a-a638-4308-92c3-d92bd56df175 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:39.129593 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b416d4-dab8-4582-9d99-a3f7a41bfde3 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:39.153538 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721e207a-a638-4308-92c3-d92bd56df175 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:39.160104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-721e207a-a638-4308-92c3-d92bd56df175 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:32:39.160104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:39.160104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:39.160104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:39.160104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:39.160104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:39.160104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:39.160104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:39.160104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:39.160104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:39.160104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:39.160104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:39.160104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:39.160104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:39.160104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:39.160104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:39.164076 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:39.164076 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:39.164076 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:39.164076 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:39.164076 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:39.164076 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:39.164076 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:39.164076 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:39.164076 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:39.164076 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:39.164076 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:39.164076 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:39.164076 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:39.164076 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:39.164076 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:39.164076 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:39.167434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:39.167434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:39.167434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:39.167434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:39.167434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:39.167434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:39.167434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:39.167434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:39.167434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:39.167434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:39.167434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:32:39.167434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:32:39.167434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 189/372] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Sep 22 02:32:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:39.167434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e207a-a638-4308-92c3-d92bd56df175 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:39.266975 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b416d4-dab8-4582-9d99-a3f7a41bfde3 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:39.309909 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e207a-a638-4308-92c3-d92bd56df175 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:39.312889 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:39.353821 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d94a451-a159-429f-9942-d7d248475c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:39.354682 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d94a451-a159-429f-9942-d7d248475c39 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:39.355057 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d94a451-a159-429f-9942-d7d248475c39 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:39.355329 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d94a451-a159-429f-9942-d7d248475c39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:39.356107 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d94a451-a159-429f-9942-d7d248475c39 None admin] RBAC: Adding query filter params (group.id=2c48871136ff46089e386a696762e416, role.id=adec3004b56243fa923bd9f0082a2728, scope.project.id=d861cd3fc0a047d6bf06a6250256ee77) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:39.356427 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d94a451-a159-429f-9942-d7d248475c39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:39.357927 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d94a451-a159-429f-9942-d7d248475c39 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:39.367537 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 184/373] 10.222.0.20 () {56 vars in 1376 bytes} [Thu Sep 22 02:32:39 2022] GET /identity/v3/role_assignments?group.id=2c48871136ff46089e386a696762e416&scope.project.id=d861cd3fc0a047d6bf06a6250256ee77&role.id=adec3004b56243fa923bd9f0082a2728 => generated 260 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:39.373581 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d94a451-a159-429f-9942-d7d248475c39 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:39.379500 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25982666-0c1a-4f4f-b4a4-811f3692e89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:39.381922 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25982666-0c1a-4f4f-b4a4-811f3692e89a None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:39.382612 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25982666-0c1a-4f4f-b4a4-811f3692e89a None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:39.383162 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25982666-0c1a-4f4f-b4a4-811f3692e89a None admin] PATH_INFO: `/v3/groups/2c48871136ff46089e386a696762e416` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:39.385347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25982666-0c1a-4f4f-b4a4-811f3692e89a None admin] RBAC: Authorizing `identity:get_group(group_id=2c48871136ff46089e386a696762e416)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:39.387407 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25982666-0c1a-4f4f-b4a4-811f3692e89a None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:39.390005 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 190/374] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Sep 22 02:32:39 2022] GET /identity/v3/groups/2c48871136ff46089e386a696762e416 => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:39.391637 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6f447-43df-46c3-95f2-0dfaa583e1ad None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:39.393144 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6f447-43df-46c3-95f2-0dfaa583e1ad None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:39.393684 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6f447-43df-46c3-95f2-0dfaa583e1ad None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:39.395554 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 191/375] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:39.396645 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6f447-43df-46c3-95f2-0dfaa583e1ad None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:39.421991 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:39.601183 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6f447-43df-46c3-95f2-0dfaa583e1ad None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:39.601373 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d94a451-a159-429f-9942-d7d248475c39 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:39.652148 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e6951-9dfb-46d4-ad26-30bc4f18ecb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:39.652706 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e6951-9dfb-46d4-ad26-30bc4f18ecb2 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:39.652985 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e6951-9dfb-46d4-ad26-30bc4f18ecb2 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:39.653322 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e6951-9dfb-46d4-ad26-30bc4f18ecb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:39.654189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e6951-9dfb-46d4-ad26-30bc4f18ecb2 None admin] RBAC: Adding query filter params (group.id=2c48871136ff46089e386a696762e416, role.id=83efac3b92eb44e783f4924c9d9ee842, scope.project.id=d861cd3fc0a047d6bf06a6250256ee77) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:39.654539 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e6951-9dfb-46d4-ad26-30bc4f18ecb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:39.656055 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e6951-9dfb-46d4-ad26-30bc4f18ecb2 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:39.662867 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 192/376] 10.222.0.20 () {56 vars in 1376 bytes} [Thu Sep 22 02:32:39 2022] GET /identity/v3/role_assignments?group.id=2c48871136ff46089e386a696762e416&scope.project.id=d861cd3fc0a047d6bf06a6250256ee77&role.id=83efac3b92eb44e783f4924c9d9ee842 => generated 260 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:39.664056 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa2596-2e06-4734-ad18-223e8d090925 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:39.664388 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa2596-2e06-4734-ad18-223e8d090925 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:39.664667 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa2596-2e06-4734-ad18-223e8d090925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:39.671694 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85dfff-cedc-4077-9a29-ed8c2e888cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:39.673983 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85dfff-cedc-4077-9a29-ed8c2e888cc1 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:39.674938 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85dfff-cedc-4077-9a29-ed8c2e888cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:39.675692 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85dfff-cedc-4077-9a29-ed8c2e888cc1 None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:39.677248 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b85dfff-cedc-4077-9a29-ed8c2e888cc1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:39.678095 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b85dfff-cedc-4077-9a29-ed8c2e888cc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:39.683113 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b85dfff-cedc-4077-9a29-ed8c2e888cc1 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:39.686488 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-68aa2596-2e06-4734-ad18-223e8d090925 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:39.687849 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-68aa2596-2e06-4734-ad18-223e8d090925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:39.704435 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 185/377] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:32:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:39.711847 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85dfff-cedc-4077-9a29-ed8c2e888cc1 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:39.776077 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 193/378] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:32:39.814965 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:39.938336 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85dfff-cedc-4077-9a29-ed8c2e888cc1 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:40.015220 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32789f5-e741-4e7f-b106-bdf459c07c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:40.016755 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32789f5-e741-4e7f-b106-bdf459c07c0e None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:40.017500 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32789f5-e741-4e7f-b106-bdf459c07c0e None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:40.018218 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32789f5-e741-4e7f-b106-bdf459c07c0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:40.185577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32789f5-e741-4e7f-b106-bdf459c07c0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:40.188210 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32789f5-e741-4e7f-b106-bdf459c07c0e None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:40.211591 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa2596-2e06-4734-ad18-223e8d090925 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:40.222855 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:40.331368 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:40.558431 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 186/379] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:32:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 848 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 22 02:32:40.564681 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32789f5-e741-4e7f-b106-bdf459c07c0e None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:40.588468 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa2596-2e06-4734-ad18-223e8d090925 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:40.693492 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd84ac01-63b6-48ed-8b58-19e8f012371e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:40.694626 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84ac01-63b6-48ed-8b58-19e8f012371e None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:40.695236 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84ac01-63b6-48ed-8b58-19e8f012371e None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:40.695725 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84ac01-63b6-48ed-8b58-19e8f012371e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:40.732733 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd84ac01-63b6-48ed-8b58-19e8f012371e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:40.751169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-fd84ac01-63b6-48ed-8b58-19e8f012371e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 02:32:40.751169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:40.751169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:40.751169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:40.751169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:40.751169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:40.751169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:40.751169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:40.751169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:40.751169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:40.751169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:40.751169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:40.751169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:40.751169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:40.751169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:40.751169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:40.752155 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:40.752155 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:40.752155 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:40.752155 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:40.752155 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:40.752155 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:40.752155 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:40.752155 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:40.752155 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:40.752155 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:40.752155 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:40.752155 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:40.752155 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:40.752155 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:40.752155 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:40.752155 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:40.752953 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:40.752953 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:40.752953 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:40.752953 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:40.752953 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:40.752953 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:40.752953 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:40.752953 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:40.752953 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:40.752953 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:40.752953 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 02:32:40.752953 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:32:40.762916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 194/380] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Sep 22 02:32:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 548 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:40.769435 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32789f5-e741-4e7f-b106-bdf459c07c0e None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:40.776549 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd84ac01-63b6-48ed-8b58-19e8f012371e None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:40.904521 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57614bb5-6c50-4b25-b833-5cfe7f9d21b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:40.906066 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57614bb5-6c50-4b25-b833-5cfe7f9d21b2 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:40.906805 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57614bb5-6c50-4b25-b833-5cfe7f9d21b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:40.907676 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57614bb5-6c50-4b25-b833-5cfe7f9d21b2 None admin] PATH_INFO: `/v3/projects/f92fe1b089244bf4b21f4f98339aa81d` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:40.915583 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57614bb5-6c50-4b25-b833-5cfe7f9d21b2 None admin] RBAC: Authorizing `identity:get_project(project_id=f92fe1b089244bf4b21f4f98339aa81d)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:40.915583 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57614bb5-6c50-4b25-b833-5cfe7f9d21b2 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:40.915583 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 187/381] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:32:40 2022] GET /identity/v3/projects/f92fe1b089244bf4b21f4f98339aa81d => generated 344 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:40.925365 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57614bb5-6c50-4b25-b833-5cfe7f9d21b2 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:40.970788 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:41.006885 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:41.094509 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd84ac01-63b6-48ed-8b58-19e8f012371e None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:41.166970 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cec648-df6a-4833-a812-c8cdcff13680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:41.168421 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cec648-df6a-4833-a812-c8cdcff13680 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:41.169192 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cec648-df6a-4833-a812-c8cdcff13680 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:41.169762 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cec648-df6a-4833-a812-c8cdcff13680 None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:41.171149 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cec648-df6a-4833-a812-c8cdcff13680 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:41.171951 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cec648-df6a-4833-a812-c8cdcff13680 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:41.174516 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cec648-df6a-4833-a812-c8cdcff13680 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:41.181402 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57614bb5-6c50-4b25-b833-5cfe7f9d21b2 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:41.202684 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 195/382] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:32:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 424 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:41.204618 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cec648-df6a-4833-a812-c8cdcff13680 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:41.220019 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4107a1-68f1-4e1b-8b87-254765b88e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:41.221383 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4107a1-68f1-4e1b-8b87-254765b88e02 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:41.221866 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4107a1-68f1-4e1b-8b87-254765b88e02 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:41.222367 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4107a1-68f1-4e1b-8b87-254765b88e02 None admin] PATH_INFO: `/v3/groups/0c4aa1bea31a44159a2b8a7cf69fa0a6` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:41.229405 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4107a1-68f1-4e1b-8b87-254765b88e02 None admin] RBAC: Authorizing `identity:get_group(group_id=0c4aa1bea31a44159a2b8a7cf69fa0a6)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:41.231347 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4107a1-68f1-4e1b-8b87-254765b88e02 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:41.233630 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 188/383] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Sep 22 02:32:40 2022] GET /identity/v3/groups/0c4aa1bea31a44159a2b8a7cf69fa0a6 => generated 229 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:41.239844 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4107a1-68f1-4e1b-8b87-254765b88e02 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:41.341477 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:41.408928 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4107a1-68f1-4e1b-8b87-254765b88e02 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:41.484720 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cec648-df6a-4833-a812-c8cdcff13680 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:41.516181 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:41.529739 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2703056f-b20c-4c5e-b0d6-909d7b488b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:41.530768 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703056f-b20c-4c5e-b0d6-909d7b488b43 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:41.531319 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703056f-b20c-4c5e-b0d6-909d7b488b43 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:41.531817 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703056f-b20c-4c5e-b0d6-909d7b488b43 None admin] PATH_INFO: `/v3/groups/2c48871136ff46089e386a696762e416` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:41.532061 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1c932-b48f-4c06-ab6e-a8e513d936a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:41.533120 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1c932-b48f-4c06-ab6e-a8e513d936a4 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:41.533614 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1c932-b48f-4c06-ab6e-a8e513d936a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:41.534044 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2703056f-b20c-4c5e-b0d6-909d7b488b43 None admin] RBAC: Authorizing `identity:get_group(group_id=2c48871136ff46089e386a696762e416)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:41.534168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1c932-b48f-4c06-ab6e-a8e513d936a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:41.535851 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b1c932-b48f-4c06-ab6e-a8e513d936a4 None admin] RBAC: Adding query filter params (group.id=0c4aa1bea31a44159a2b8a7cf69fa0a6, role.id=31a687548f55493b8ea666fe7e350303, scope.project.id=f92fe1b089244bf4b21f4f98339aa81d) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:41.536133 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2703056f-b20c-4c5e-b0d6-909d7b488b43 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:41.536406 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b1c932-b48f-4c06-ab6e-a8e513d936a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:41.538704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 196/384] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Sep 22 02:32:41 2022] GET /identity/v3/groups/2c48871136ff46089e386a696762e416 => generated 226 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:41.538894 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b1c932-b48f-4c06-ab6e-a8e513d936a4 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:41.545249 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2703056f-b20c-4c5e-b0d6-909d7b488b43 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:41.553906 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 189/385] 10.222.0.20 () {56 vars in 1376 bytes} [Thu Sep 22 02:32:41 2022] GET /identity/v3/role_assignments?group.id=0c4aa1bea31a44159a2b8a7cf69fa0a6&scope.project.id=f92fe1b089244bf4b21f4f98339aa81d&role.id=31a687548f55493b8ea666fe7e350303 => generated 260 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:32:41.674356 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2703056f-b20c-4c5e-b0d6-909d7b488b43 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:41.726802 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b0fc0b-3d68-4164-af53-6e28fe73bf17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:41.728520 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0fc0b-3d68-4164-af53-6e28fe73bf17 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:41.729028 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0fc0b-3d68-4164-af53-6e28fe73bf17 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:41.729518 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0fc0b-3d68-4164-af53-6e28fe73bf17 None admin] PATH_INFO: `/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:41.734042 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b0fc0b-3d68-4164-af53-6e28fe73bf17 None admin] RBAC: Authorizing `identity:get_project(project_id=6a5cfaff6dc7495f9a0648c76c9f6eb7)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:41.737546 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b0fc0b-3d68-4164-af53-6e28fe73bf17 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:41.741683 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 197/386] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:32:41 2022] GET /identity/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:32:41.749339 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1c932-b48f-4c06-ab6e-a8e513d936a4 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:41.978379 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542fdec-17bf-4efa-96ee-ce92e2e29807 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:41.979013 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542fdec-17bf-4efa-96ee-ce92e2e29807 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:41.979509 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542fdec-17bf-4efa-96ee-ce92e2e29807 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:41.981412 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 198/387] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:41.989926 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9919f-3ac9-40f3-99b9-d59235b38f6c None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:41.990316 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9919f-3ac9-40f3-99b9-d59235b38f6c None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:41.990785 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9919f-3ac9-40f3-99b9-d59235b38f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:42.005552 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1c932-b48f-4c06-ab6e-a8e513d936a4 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:42.049648 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-23a9919f-3ac9-40f3-99b9-d59235b38f6c None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:42.051301 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-23a9919f-3ac9-40f3-99b9-d59235b38f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:42.089151 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a16357-b763-49e8-9166-beab13f1ab94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:42.089858 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a16357-b763-49e8-9166-beab13f1ab94 None admin] REQUEST_METHOD: `PUT` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:42.090146 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a16357-b763-49e8-9166-beab13f1ab94 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:42.090429 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a16357-b763-49e8-9166-beab13f1ab94 None admin] PATH_INFO: `/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7/groups/2c48871136ff46089e386a696762e416/roles/adec3004b56243fa923bd9f0082a2728` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:42.097353 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a16357-b763-49e8-9166-beab13f1ab94 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a5cfaff6dc7495f9a0648c76c9f6eb7, group_id=2c48871136ff46089e386a696762e416, role_id=adec3004b56243fa923bd9f0082a2728)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:42.098949 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a16357-b763-49e8-9166-beab13f1ab94 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:42.122381 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:42.125185 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 190/388] 10.222.0.20 () {58 vars in 1442 bytes} [Thu Sep 22 02:32:41 2022] PUT /identity/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7/groups/2c48871136ff46089e386a696762e416/roles/adec3004b56243fa923bd9f0082a2728 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 02:32:42.170076 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:42.173242 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 199/389] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:32:42.563335 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a16357-b763-49e8-9166-beab13f1ab94 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:42.764395 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a16357-b763-49e8-9166-beab13f1ab94 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:42.848885 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6fc923-0ca6-440d-a8dc-354145003532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:42.849967 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6fc923-0ca6-440d-a8dc-354145003532 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:42.850453 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6fc923-0ca6-440d-a8dc-354145003532 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:42.850956 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6fc923-0ca6-440d-a8dc-354145003532 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:42.885318 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6fc923-0ca6-440d-a8dc-354145003532 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:42.896186 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-cf6fc923-0ca6-440d-a8dc-354145003532 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 02:32:42.896186 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:42.896186 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:42.896186 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:42.896186 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:42.896186 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:42.896186 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:42.896186 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:42.896186 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:42.896186 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:42.896186 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:42.896186 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:42.896186 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:42.896186 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:42.896186 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:42.896186 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:42.897291 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:42.897291 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:42.897291 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:42.897291 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:42.897291 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:42.897291 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:42.897291 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:42.897291 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:42.897291 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:42.897291 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:42.897291 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:42.897291 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:42.897291 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:42.897291 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:42.897291 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:42.897291 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:42.898305 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:42.898305 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:42.898305 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:42.898305 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:42.898305 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:42.898305 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:42.898305 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:42.898305 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:42.898305 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:42.898305 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:42.898305 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 02:32:42.898305 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:32:42.902971 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 191/390] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Sep 22 02:32:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:42.905726 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a9919f-3ac9-40f3-99b9-d59235b38f6c None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:42.944598 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcbb78-0f2b-489f-9957-c70417599fcd None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:42.945087 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcbb78-0f2b-489f-9957-c70417599fcd None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:42.945585 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcbb78-0f2b-489f-9957-c70417599fcd None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:42.950887 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 192/391] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:42.958071 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7f8ed-288b-4db7-bedb-98db5b3f05ce None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:42.958557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7f8ed-288b-4db7-bedb-98db5b3f05ce None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:42.959078 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7f8ed-288b-4db7-bedb-98db5b3f05ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:43.013884 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-f3b7f8ed-288b-4db7-bedb-98db5b3f05ce None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:43.018872 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-f3b7f8ed-288b-4db7-bedb-98db5b3f05ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:43.183254 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a9919f-3ac9-40f3-99b9-d59235b38f6c None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:43.214803 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:43.237437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 193/392] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 02:32:43.239293 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1761d922-0929-4759-8907-6e8c49cac814 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:43.239586 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1761d922-0929-4759-8907-6e8c49cac814 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:43.239887 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1761d922-0929-4759-8907-6e8c49cac814 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:43.242954 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 194/393] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:43.249400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c5220-0901-410a-945e-2bd5bed7e0f5 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:43.249819 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c5220-0901-410a-945e-2bd5bed7e0f5 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:43.250206 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c5220-0901-410a-945e-2bd5bed7e0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:43.269903 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a240ec-750c-4a4a-98f4-7942276621bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:43.271037 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a240ec-750c-4a4a-98f4-7942276621bc None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:43.271790 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a240ec-750c-4a4a-98f4-7942276621bc None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:43.272443 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a240ec-750c-4a4a-98f4-7942276621bc None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:43.273602 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a240ec-750c-4a4a-98f4-7942276621bc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:43.274516 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a240ec-750c-4a4a-98f4-7942276621bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:43.276982 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a240ec-750c-4a4a-98f4-7942276621bc None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:43.282667 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-385c5220-0901-410a-945e-2bd5bed7e0f5 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:43.284789 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-385c5220-0901-410a-945e-2bd5bed7e0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:43.292885 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 200/394] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:32:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:43.298595 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a240ec-750c-4a4a-98f4-7942276621bc None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:43.376114 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:43.451192 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a240ec-750c-4a4a-98f4-7942276621bc None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:43.483407 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 195/395] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 02:32:43.503674 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4080650-e4dd-447d-94b8-599a66125b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:43.504775 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4080650-e4dd-447d-94b8-599a66125b73 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:43.505306 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4080650-e4dd-447d-94b8-599a66125b73 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:43.505806 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4080650-e4dd-447d-94b8-599a66125b73 None admin] PATH_INFO: `/v3/groups/2c48871136ff46089e386a696762e416` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:43.511777 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4080650-e4dd-447d-94b8-599a66125b73 None admin] RBAC: Authorizing `identity:get_group(group_id=2c48871136ff46089e386a696762e416)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:43.513600 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4080650-e4dd-447d-94b8-599a66125b73 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:43.515802 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 201/396] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Sep 22 02:32:43 2022] GET /identity/v3/groups/2c48871136ff46089e386a696762e416 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:32:43.521637 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c5220-0901-410a-945e-2bd5bed7e0f5 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:43.690374 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c5220-0901-410a-945e-2bd5bed7e0f5 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:43.720337 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:43.744868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdbbf70-2725-4f40-961f-be850c95c01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:43.745529 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbbf70-2725-4f40-961f-be850c95c01e None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:43.745817 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbbf70-2725-4f40-961f-be850c95c01e None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:43.746097 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbbf70-2725-4f40-961f-be850c95c01e None admin] PATH_INFO: `/v3/projects/d861cd3fc0a047d6bf06a6250256ee77` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:43.747279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdbbf70-2725-4f40-961f-be850c95c01e None admin] RBAC: Authorizing `identity:get_project(project_id=d861cd3fc0a047d6bf06a6250256ee77)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:43.748750 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdbbf70-2725-4f40-961f-be850c95c01e None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:43.750717 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 196/397] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:32:43 2022] GET /identity/v3/projects/d861cd3fc0a047d6bf06a6250256ee77 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:43.756673 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4080650-e4dd-447d-94b8-599a66125b73 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:43.784149 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdbbf70-2725-4f40-961f-be850c95c01e None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:43.911907 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4080650-e4dd-447d-94b8-599a66125b73 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:43.938277 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdbbf70-2725-4f40-961f-be850c95c01e None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:43.939229 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:43.964827 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b74663-a634-47be-97eb-ece54c1b62a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:43.965770 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b74663-a634-47be-97eb-ece54c1b62a0 None admin] REQUEST_METHOD: `PUT` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:43.966237 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b74663-a634-47be-97eb-ece54c1b62a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:43.966656 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b74663-a634-47be-97eb-ece54c1b62a0 None admin] PATH_INFO: `/v3/projects/d861cd3fc0a047d6bf06a6250256ee77/groups/2c48871136ff46089e386a696762e416/roles/adec3004b56243fa923bd9f0082a2728` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:43.969884 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b74663-a634-47be-97eb-ece54c1b62a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=d861cd3fc0a047d6bf06a6250256ee77, group_id=2c48871136ff46089e386a696762e416, role_id=adec3004b56243fa923bd9f0082a2728)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:43.972259 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b74663-a634-47be-97eb-ece54c1b62a0 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:43.987088 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f993d-3523-455f-a53b-0ebf5d47fec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:43.988007 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f993d-3523-455f-a53b-0ebf5d47fec1 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:43.988385 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f993d-3523-455f-a53b-0ebf5d47fec1 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:43.989017 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f993d-3523-455f-a53b-0ebf5d47fec1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:43.995124 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 202/398] 10.222.0.20 () {58 vars in 1442 bytes} [Thu Sep 22 02:32:43 2022] PUT /identity/v3/projects/d861cd3fc0a047d6bf06a6250256ee77/groups/2c48871136ff46089e386a696762e416/roles/adec3004b56243fa923bd9f0082a2728 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 02:32:43.997669 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b74663-a634-47be-97eb-ece54c1b62a0 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:44.026254 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492f993d-3523-455f-a53b-0ebf5d47fec1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:44.033117 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-492f993d-3523-455f-a53b-0ebf5d47fec1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 02:32:44.033117 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:44.033117 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:44.033117 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:44.033117 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:44.033117 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:44.033117 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:44.033117 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:44.033117 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:44.033117 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:44.033117 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:44.033117 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:44.033117 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:44.033117 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:44.033117 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:44.033117 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:44.034101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:44.034101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:44.034101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:44.034101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:44.034101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:44.034101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:44.034101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:44.034101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:44.034101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:44.034101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:44.034101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:44.034101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:44.034101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:44.034101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:44.034101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:44.034101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:44.034923 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:44.034923 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:44.034923 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:44.034923 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:44.034923 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:44.034923 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:44.034923 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:44.034923 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:44.034923 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:44.034923 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:44.034923 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 02:32:44.034923 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:32:44.036764 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 197/399] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:32:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:44.043086 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f993d-3523-455f-a53b-0ebf5d47fec1 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:44.154543 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:44.247745 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b74663-a634-47be-97eb-ece54c1b62a0 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:44.265167 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f993d-3523-455f-a53b-0ebf5d47fec1 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:44.288931 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e523abc-0e85-4a00-9876-fc3745dd3d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:44.289583 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e523abc-0e85-4a00-9876-fc3745dd3d44 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:44.289895 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e523abc-0e85-4a00-9876-fc3745dd3d44 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:44.290753 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e523abc-0e85-4a00-9876-fc3745dd3d44 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:44.303158 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e523abc-0e85-4a00-9876-fc3745dd3d44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:44.308718 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-9e523abc-0e85-4a00-9876-fc3745dd3d44 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 02:32:44.308718 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:44.308718 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:44.308718 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:44.308718 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:44.308718 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:44.308718 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:44.308718 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:44.308718 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:44.308718 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:44.308718 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:44.308718 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:44.308718 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:44.308718 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:44.308718 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:44.308718 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:44.309984 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:44.309984 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:44.309984 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:44.309984 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:44.309984 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:44.309984 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:44.309984 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:44.309984 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:44.309984 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:44.309984 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:44.309984 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:44.309984 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:44.309984 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:44.309984 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:44.309984 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:44.309984 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:44.311374 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:44.311374 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:44.311374 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:44.311374 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:44.311374 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:44.311374 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:44.311374 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:44.311374 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:44.311374 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:44.311374 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:44.311374 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 02:32:44.311374 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:32:44.311374 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 203/400] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:32:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:44.316612 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e523abc-0e85-4a00-9876-fc3745dd3d44 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:44.338764 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44e58c-0827-4c1d-9e73-e37475ac4554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:44.339839 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44e58c-0827-4c1d-9e73-e37475ac4554 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:44.340378 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44e58c-0827-4c1d-9e73-e37475ac4554 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:44.340853 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44e58c-0827-4c1d-9e73-e37475ac4554 None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:44.341753 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec44e58c-0827-4c1d-9e73-e37475ac4554 None admin] RBAC: Adding query filter params (name=member) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:44.342261 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec44e58c-0827-4c1d-9e73-e37475ac4554 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:44.343913 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec44e58c-0827-4c1d-9e73-e37475ac4554 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:44.354279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 198/401] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Sep 22 02:32:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:44.361341 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44e58c-0827-4c1d-9e73-e37475ac4554 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:44.366564 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:44.442255 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e523abc-0e85-4a00-9876-fc3745dd3d44 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:44.485560 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d917d-b299-40af-8a4f-c3e8bb03b45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:44.486259 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d917d-b299-40af-8a4f-c3e8bb03b45d None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:44.487599 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d917d-b299-40af-8a4f-c3e8bb03b45d None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:44.487599 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d917d-b299-40af-8a4f-c3e8bb03b45d None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:44.487842 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d917d-b299-40af-8a4f-c3e8bb03b45d None admin] RBAC: Adding query filter params (name=member) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:44.488097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d917d-b299-40af-8a4f-c3e8bb03b45d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:44.489516 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d917d-b299-40af-8a4f-c3e8bb03b45d None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:44.499944 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 204/402] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Sep 22 02:32:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:44.505918 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d917d-b299-40af-8a4f-c3e8bb03b45d None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:44.539501 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44e58c-0827-4c1d-9e73-e37475ac4554 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:44.552204 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:44.583057 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61102894-1b5c-4f73-bbac-904540793ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:44.584117 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61102894-1b5c-4f73-bbac-904540793ad4 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:44.584625 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61102894-1b5c-4f73-bbac-904540793ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:44.585259 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61102894-1b5c-4f73-bbac-904540793ad4 None admin] PATH_INFO: `/v3/groups/2c48871136ff46089e386a696762e416` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:44.587236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61102894-1b5c-4f73-bbac-904540793ad4 None admin] RBAC: Authorizing `identity:get_group(group_id=2c48871136ff46089e386a696762e416)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:44.588971 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61102894-1b5c-4f73-bbac-904540793ad4 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:44.591211 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 199/403] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Sep 22 02:32:44 2022] GET /identity/v3/groups/2c48871136ff46089e386a696762e416 => generated 226 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:32:44.598899 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61102894-1b5c-4f73-bbac-904540793ad4 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:44.653496 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d917d-b299-40af-8a4f-c3e8bb03b45d None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:44.666137 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:44.692746 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1af74-b8a8-4cdd-b8eb-2c8f0337540e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:44.693512 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1af74-b8a8-4cdd-b8eb-2c8f0337540e None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:44.693865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1af74-b8a8-4cdd-b8eb-2c8f0337540e None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:44.694211 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1af74-b8a8-4cdd-b8eb-2c8f0337540e None admin] PATH_INFO: `/v3/groups/2c48871136ff46089e386a696762e416` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:44.695792 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf1af74-b8a8-4cdd-b8eb-2c8f0337540e None admin] RBAC: Authorizing `identity:get_group(group_id=2c48871136ff46089e386a696762e416)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:44.698023 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf1af74-b8a8-4cdd-b8eb-2c8f0337540e None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:44.700418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 205/404] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Sep 22 02:32:44 2022] GET /identity/v3/groups/2c48871136ff46089e386a696762e416 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:44.706661 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1af74-b8a8-4cdd-b8eb-2c8f0337540e None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:44.827952 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1af74-b8a8-4cdd-b8eb-2c8f0337540e None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:44.866087 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32175963-c230-472d-8314-5706fb70eae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:44.867314 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32175963-c230-472d-8314-5706fb70eae8 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:44.868137 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32175963-c230-472d-8314-5706fb70eae8 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:44.868822 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32175963-c230-472d-8314-5706fb70eae8 None admin] PATH_INFO: `/v3/projects/d861cd3fc0a047d6bf06a6250256ee77` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:44.870969 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32175963-c230-472d-8314-5706fb70eae8 None admin] RBAC: Authorizing `identity:get_project(project_id=d861cd3fc0a047d6bf06a6250256ee77)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:44.872687 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32175963-c230-472d-8314-5706fb70eae8 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:44.874920 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 206/405] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:32:44 2022] GET /identity/v3/projects/d861cd3fc0a047d6bf06a6250256ee77 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:44.880768 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32175963-c230-472d-8314-5706fb70eae8 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:44.935530 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:45.002380 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32175963-c230-472d-8314-5706fb70eae8 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:45.014228 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61102894-1b5c-4f73-bbac-904540793ad4 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:45.042253 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ce7e7-741c-4058-99d7-fce46c9c22d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:45.043298 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ce7e7-741c-4058-99d7-fce46c9c22d9 None admin] REQUEST_METHOD: `PUT` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:45.043821 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ce7e7-741c-4058-99d7-fce46c9c22d9 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:45.044300 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ce7e7-741c-4058-99d7-fce46c9c22d9 None admin] PATH_INFO: `/v3/projects/d861cd3fc0a047d6bf06a6250256ee77/groups/2c48871136ff46089e386a696762e416/roles/83efac3b92eb44e783f4924c9d9ee842` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:45.046594 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548ce7e7-741c-4058-99d7-fce46c9c22d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=d861cd3fc0a047d6bf06a6250256ee77, group_id=2c48871136ff46089e386a696762e416, role_id=83efac3b92eb44e783f4924c9d9ee842)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:45.048357 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548ce7e7-741c-4058-99d7-fce46c9c22d9 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:45.064598 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 207/406] 10.222.0.20 () {58 vars in 1442 bytes} [Thu Sep 22 02:32:44 2022] PUT /identity/v3/projects/d861cd3fc0a047d6bf06a6250256ee77/groups/2c48871136ff46089e386a696762e416/roles/83efac3b92eb44e783f4924c9d9ee842 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 02:32:45.105203 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f01a3-43dc-4267-bf99-afc407746244 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:45.105757 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f01a3-43dc-4267-bf99-afc407746244 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:45.106215 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f01a3-43dc-4267-bf99-afc407746244 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:45.108280 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 208/407] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:45.114850 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da533f-a819-4d2a-9ad8-d11c9d662f1f None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:45.115615 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da533f-a819-4d2a-9ad8-d11c9d662f1f None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:45.116083 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da533f-a819-4d2a-9ad8-d11c9d662f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:45.127998 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd885009-3e72-4e72-996b-531e182e9386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:45.128656 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd885009-3e72-4e72-996b-531e182e9386 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:45.128938 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd885009-3e72-4e72-996b-531e182e9386 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:45.129216 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd885009-3e72-4e72-996b-531e182e9386 None admin] PATH_INFO: `/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:45.131274 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd885009-3e72-4e72-996b-531e182e9386 None admin] RBAC: Authorizing `identity:get_project(project_id=6a5cfaff6dc7495f9a0648c76c9f6eb7)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:45.136570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd885009-3e72-4e72-996b-531e182e9386 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:45.143316 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 200/408] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:32:44 2022] GET /identity/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7 => generated 298 bytes in 545 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:32:45.147291 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd885009-3e72-4e72-996b-531e182e9386 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:45.152040 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-04da533f-a819-4d2a-9ad8-d11c9d662f1f None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:45.153480 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-04da533f-a819-4d2a-9ad8-d11c9d662f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:45.214988 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:45.319004 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd885009-3e72-4e72-996b-531e182e9386 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:45.327666 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 209/409] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 02:32:45.380156 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c340623c-e009-48f2-980e-495132f4b00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:45.380967 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340623c-e009-48f2-980e-495132f4b00f None admin] REQUEST_METHOD: `PUT` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:45.381443 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340623c-e009-48f2-980e-495132f4b00f None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:45.381884 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340623c-e009-48f2-980e-495132f4b00f None admin] PATH_INFO: `/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7/groups/2c48871136ff46089e386a696762e416/roles/83efac3b92eb44e783f4924c9d9ee842` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:45.388267 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c340623c-e009-48f2-980e-495132f4b00f None admin] RBAC: Authorizing `identity:create_grant(project_id=6a5cfaff6dc7495f9a0648c76c9f6eb7, group_id=2c48871136ff46089e386a696762e416, role_id=83efac3b92eb44e783f4924c9d9ee842)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:45.390627 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c340623c-e009-48f2-980e-495132f4b00f None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:45.413873 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 201/410] 10.222.0.20 () {58 vars in 1442 bytes} [Thu Sep 22 02:32:45 2022] PUT /identity/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7/groups/2c48871136ff46089e386a696762e416/roles/83efac3b92eb44e783f4924c9d9ee842 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 02:32:45.709734 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de53e4-d86f-4d33-8963-b6d63751d72e None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:45.709924 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de53e4-d86f-4d33-8963-b6d63751d72e None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:45.710196 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de53e4-d86f-4d33-8963-b6d63751d72e None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:45.712459 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 210/411] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:45.718444 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0c69f-596d-4e60-bf8b-19ac7692411c None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:45.718733 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0c69f-596d-4e60-bf8b-19ac7692411c None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:45.719039 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0c69f-596d-4e60-bf8b-19ac7692411c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:45.739857 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-b7f0c69f-596d-4e60-bf8b-19ac7692411c None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:45.741118 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-b7f0c69f-596d-4e60-bf8b-19ac7692411c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:45.830949 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 202/412] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 02:32:45.848681 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de53e4-d86f-4d33-8963-b6d63751d72e None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:45.903535 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:46.085557 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de53e4-d86f-4d33-8963-b6d63751d72e None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:46.176006 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4ad77-f904-42ba-b454-8066b882b1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:46.177124 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4ad77-f904-42ba-b454-8066b882b1ef None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:46.177626 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4ad77-f904-42ba-b454-8066b882b1ef None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:46.178121 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4ad77-f904-42ba-b454-8066b882b1ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:46.205120 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0c69f-596d-4e60-bf8b-19ac7692411c None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:46.207762 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4ad77-f904-42ba-b454-8066b882b1ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:46.218108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-06e4ad77-f904-42ba-b454-8066b882b1ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:32:46.218108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:46.218108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:46.218108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:46.218108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:46.218108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:46.218108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:46.218108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:46.218108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:46.218108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:46.218108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:46.218108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:46.218108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:46.218108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:46.218108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:46.218108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:46.219105 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:46.219105 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:46.219105 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:46.219105 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:46.219105 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:46.219105 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:46.219105 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:46.219105 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:46.219105 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:46.219105 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:46.219105 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:46.219105 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:46.219105 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:46.219105 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:46.219105 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:46.219105 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:46.219932 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:46.219932 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:46.219932 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:46.219932 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:46.219932 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:46.219932 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:46.219932 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:46.219932 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:46.219932 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:46.219932 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:46.219932 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:32:46.219932 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:32:46.222084 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 211/413] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Sep 22 02:32:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:46.230795 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4ad77-f904-42ba-b454-8066b882b1ef None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:46.312204 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:46.382700 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:46.392762 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4ad77-f904-42ba-b454-8066b882b1ef None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:46.431317 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c07bd-3d69-41d1-84af-b20995871b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:46.431947 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c07bd-3d69-41d1-84af-b20995871b7c None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:46.432233 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c07bd-3d69-41d1-84af-b20995871b7c None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:46.432512 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c07bd-3d69-41d1-84af-b20995871b7c None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:46.433194 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147c07bd-3d69-41d1-84af-b20995871b7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:46.433525 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147c07bd-3d69-41d1-84af-b20995871b7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:46.434936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147c07bd-3d69-41d1-84af-b20995871b7c None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:46.449702 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 212/414] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:32:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:46.459699 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c07bd-3d69-41d1-84af-b20995871b7c None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:46.532548 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0c69f-596d-4e60-bf8b-19ac7692411c None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:46.588298 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2deae8f-7436-4511-b31e-17363fab9327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:46.588971 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2deae8f-7436-4511-b31e-17363fab9327 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:46.589254 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2deae8f-7436-4511-b31e-17363fab9327 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:46.589541 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2deae8f-7436-4511-b31e-17363fab9327 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:46.603231 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2deae8f-7436-4511-b31e-17363fab9327 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:46.609256 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-d2deae8f-7436-4511-b31e-17363fab9327 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 02:32:46.609256 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:46.609256 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:46.609256 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:46.609256 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:46.609256 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:46.609256 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:46.609256 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:46.609256 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:46.609256 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:46.609256 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:46.609256 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:46.609256 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:46.609256 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:46.609256 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:46.609256 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:46.610225 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:46.610225 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:46.610225 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:46.610225 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:46.610225 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:46.610225 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:46.610225 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:46.610225 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:46.610225 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:46.610225 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:46.610225 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:46.610225 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:46.610225 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:46.610225 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:46.610225 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:46.610225 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:46.611074 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:46.611074 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:46.611074 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:46.611074 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:46.611074 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:46.611074 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:46.611074 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:46.611074 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:46.611074 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:46.611074 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:46.611074 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 02:32:46.611074 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:32:46.611074 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 203/415] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Sep 22 02:32:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:46.623122 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2deae8f-7436-4511-b31e-17363fab9327 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:46.679502 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c07bd-3d69-41d1-84af-b20995871b7c None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:46.723488 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a8085f-9514-47c7-8f74-f56979c80187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:46.724617 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a8085f-9514-47c7-8f74-f56979c80187 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:46.725106 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a8085f-9514-47c7-8f74-f56979c80187 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:46.725608 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a8085f-9514-47c7-8f74-f56979c80187 None admin] PATH_INFO: `/v3/groups/0c4aa1bea31a44159a2b8a7cf69fa0a6` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:46.728897 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a8085f-9514-47c7-8f74-f56979c80187 None admin] RBAC: Authorizing `identity:get_group(group_id=0c4aa1bea31a44159a2b8a7cf69fa0a6)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:46.730617 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a8085f-9514-47c7-8f74-f56979c80187 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:46.732742 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 213/416] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Sep 22 02:32:46 2022] GET /identity/v3/groups/0c4aa1bea31a44159a2b8a7cf69fa0a6 => generated 229 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:46.741085 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a8085f-9514-47c7-8f74-f56979c80187 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:46.764107 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:46.841223 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:46.857746 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2deae8f-7436-4511-b31e-17363fab9327 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:46.863506 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a8085f-9514-47c7-8f74-f56979c80187 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:46.917131 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdad9aa-034c-4fed-974a-a12d6ac7a7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:46.917800 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdad9aa-034c-4fed-974a-a12d6ac7a7f9 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:46.918084 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdad9aa-034c-4fed-974a-a12d6ac7a7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:46.918363 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdad9aa-034c-4fed-974a-a12d6ac7a7f9 None admin] PATH_INFO: `/v3/projects/f92fe1b089244bf4b21f4f98339aa81d` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:46.919573 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdad9aa-034c-4fed-974a-a12d6ac7a7f9 None admin] RBAC: Authorizing `identity:get_project(project_id=f92fe1b089244bf4b21f4f98339aa81d)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:46.921048 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdad9aa-034c-4fed-974a-a12d6ac7a7f9 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:46.922716 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 214/417] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:32:46 2022] GET /identity/v3/projects/f92fe1b089244bf4b21f4f98339aa81d => generated 344 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:46.938704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdad9aa-034c-4fed-974a-a12d6ac7a7f9 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:46.945387 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4954b-1683-46d7-a8b4-d7d4cfc1a49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:46.946456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4954b-1683-46d7-a8b4-d7d4cfc1a49c None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:46.946973 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4954b-1683-46d7-a8b4-d7d4cfc1a49c None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:46.947449 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4954b-1683-46d7-a8b4-d7d4cfc1a49c None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:46.948647 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d4954b-1683-46d7-a8b4-d7d4cfc1a49c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:46.949181 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d4954b-1683-46d7-a8b4-d7d4cfc1a49c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:46.951792 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d4954b-1683-46d7-a8b4-d7d4cfc1a49c None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:46.980197 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 204/418] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:32:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:46.992625 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4954b-1683-46d7-a8b4-d7d4cfc1a49c None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:47.152826 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:47.203349 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdad9aa-034c-4fed-974a-a12d6ac7a7f9 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:47.295118 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4954b-1683-46d7-a8b4-d7d4cfc1a49c None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:47.313033 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f26547d-4dc0-46e9-bc5d-be7b1f9f9990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:47.314150 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26547d-4dc0-46e9-bc5d-be7b1f9f9990 None admin] REQUEST_METHOD: `PUT` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:47.314629 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26547d-4dc0-46e9-bc5d-be7b1f9f9990 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:47.315123 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26547d-4dc0-46e9-bc5d-be7b1f9f9990 None admin] PATH_INFO: `/v3/projects/f92fe1b089244bf4b21f4f98339aa81d/groups/0c4aa1bea31a44159a2b8a7cf69fa0a6/roles/31a687548f55493b8ea666fe7e350303` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:47.329418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f26547d-4dc0-46e9-bc5d-be7b1f9f9990 None admin] RBAC: Authorizing `identity:create_grant(project_id=f92fe1b089244bf4b21f4f98339aa81d, group_id=0c4aa1bea31a44159a2b8a7cf69fa0a6, role_id=31a687548f55493b8ea666fe7e350303)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:47.332798 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f26547d-4dc0-46e9-bc5d-be7b1f9f9990 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:47.355813 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d736f22-f33b-4187-b663-39547a895d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:47.356808 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d736f22-f33b-4187-b663-39547a895d3a None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:47.357373 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d736f22-f33b-4187-b663-39547a895d3a None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:47.357849 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d736f22-f33b-4187-b663-39547a895d3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:47.362905 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 215/419] 10.222.0.20 () {58 vars in 1442 bytes} [Thu Sep 22 02:32:46 2022] PUT /identity/v3/projects/f92fe1b089244bf4b21f4f98339aa81d/groups/0c4aa1bea31a44159a2b8a7cf69fa0a6/roles/31a687548f55493b8ea666fe7e350303 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 02:32:47.364758 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e42191-a68e-4cf0-9778-348ce53bbeb0 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:47.365308 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e42191-a68e-4cf0-9778-348ce53bbeb0 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:47.365829 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e42191-a68e-4cf0-9778-348ce53bbeb0 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:47.369027 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 216/420] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:47.376286 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5fc683-d4f0-47c4-8ab1-313b1d990469 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:47.376613 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5fc683-d4f0-47c4-8ab1-313b1d990469 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:47.376848 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5fc683-d4f0-47c4-8ab1-313b1d990469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:47.421293 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:47.421526 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-ec5fc683-d4f0-47c4-8ab1-313b1d990469 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:47.422429 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-ec5fc683-d4f0-47c4-8ab1-313b1d990469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:47.525746 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d736f22-f33b-4187-b663-39547a895d3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:47.529311 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d736f22-f33b-4187-b663-39547a895d3a None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:47.682680 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 217/421] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:32:47.738361 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 205/422] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:32:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 747 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 02:32:47.746881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5fc683-d4f0-47c4-8ab1-313b1d990469 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:47.884462 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:47.925083 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5fc683-d4f0-47c4-8ab1-313b1d990469 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:47.971584 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71498e55-a69e-4423-a5b8-825a969e9712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:47.972150 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71498e55-a69e-4423-a5b8-825a969e9712 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:47.972435 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71498e55-a69e-4423-a5b8-825a969e9712 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:47.972731 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71498e55-a69e-4423-a5b8-825a969e9712 None admin] PATH_INFO: `/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:47.973958 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71498e55-a69e-4423-a5b8-825a969e9712 None admin] RBAC: Authorizing `identity:get_project(project_id=6a5cfaff6dc7495f9a0648c76c9f6eb7)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:47.975466 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71498e55-a69e-4423-a5b8-825a969e9712 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:47.978113 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 218/423] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:32:47 2022] GET /identity/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7 => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:47.983877 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d736f22-f33b-4187-b663-39547a895d3a None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:48.079055 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:48.148766 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d736f22-f33b-4187-b663-39547a895d3a None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:48.201946 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc292856-28ba-4875-9ea8-dd0141b56cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:48.202654 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc292856-28ba-4875-9ea8-dd0141b56cad None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:48.202965 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc292856-28ba-4875-9ea8-dd0141b56cad None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:48.203234 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc292856-28ba-4875-9ea8-dd0141b56cad None admin] PATH_INFO: `/v3/groups/2c48871136ff46089e386a696762e416` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:48.207550 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc292856-28ba-4875-9ea8-dd0141b56cad None admin] RBAC: Authorizing `identity:get_group(group_id=2c48871136ff46089e386a696762e416)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:48.209027 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc292856-28ba-4875-9ea8-dd0141b56cad None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:48.211005 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 206/424] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Sep 22 02:32:47 2022] GET /identity/v3/groups/2c48871136ff46089e386a696762e416 => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:32:48.218390 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71498e55-a69e-4423-a5b8-825a969e9712 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:48.278306 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc292856-28ba-4875-9ea8-dd0141b56cad None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:48.453625 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71498e55-a69e-4423-a5b8-825a969e9712 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:48.565002 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce242d5-6f5a-469a-88ba-242cf453c402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:48.566671 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce242d5-6f5a-469a-88ba-242cf453c402 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:48.567559 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce242d5-6f5a-469a-88ba-242cf453c402 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:48.568602 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce242d5-6f5a-469a-88ba-242cf453c402 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:48.573202 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce242d5-6f5a-469a-88ba-242cf453c402 None admin] RBAC: Adding query filter params (group.id=2c48871136ff46089e386a696762e416, role.id=adec3004b56243fa923bd9f0082a2728, scope.project.id=6a5cfaff6dc7495f9a0648c76c9f6eb7) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:48.574287 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce242d5-6f5a-469a-88ba-242cf453c402 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:48.577974 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce242d5-6f5a-469a-88ba-242cf453c402 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:48.603836 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 219/425] 10.222.0.20 () {56 vars in 1376 bytes} [Thu Sep 22 02:32:48 2022] GET /identity/v3/role_assignments?group.id=2c48871136ff46089e386a696762e416&scope.project.id=6a5cfaff6dc7495f9a0648c76c9f6eb7&role.id=adec3004b56243fa923bd9f0082a2728 => generated 611 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:48.621343 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc292856-28ba-4875-9ea8-dd0141b56cad None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:48.689888 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:48.720933 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305cde3-a4f5-4a0f-b4f5-3cfd1eb6ba74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:48.722621 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305cde3-a4f5-4a0f-b4f5-3cfd1eb6ba74 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:48.723524 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305cde3-a4f5-4a0f-b4f5-3cfd1eb6ba74 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:48.724465 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305cde3-a4f5-4a0f-b4f5-3cfd1eb6ba74 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:48.753395 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53628bc2-d4bc-448f-912b-8755b17cd759 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:48.754307 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53628bc2-d4bc-448f-912b-8755b17cd759 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:48.755124 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53628bc2-d4bc-448f-912b-8755b17cd759 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:48.758182 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 220/426] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:32:48.764553 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87120e2d-1882-4a72-b120-e8b4780efe7f None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:48.764870 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87120e2d-1882-4a72-b120-e8b4780efe7f None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:48.765122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87120e2d-1882-4a72-b120-e8b4780efe7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:48.769279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305cde3-a4f5-4a0f-b4f5-3cfd1eb6ba74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:48.779651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-1305cde3-a4f5-4a0f-b4f5-3cfd1eb6ba74 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 02:32:48.779651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:48.779651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:48.779651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:48.779651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:48.779651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:48.779651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:48.779651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:48.779651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:48.779651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:48.779651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:48.779651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:48.779651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:48.779651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:48.779651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:48.779651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:48.780636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:48.780636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:48.780636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:48.780636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:48.780636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:48.780636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:48.780636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:48.780636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:48.780636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:48.780636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:48.780636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:48.780636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:48.780636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:48.780636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:48.780636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:48.780636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:48.781432 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:48.781432 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:48.781432 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:48.781432 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:48.781432 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:48.781432 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:48.781432 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:48.781432 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:48.781432 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:48.781432 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:48.781432 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 02:32:48.781432 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:32:48.791031 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 207/427] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Sep 22 02:32:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 506 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:48.794561 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305cde3-a4f5-4a0f-b4f5-3cfd1eb6ba74 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:48.803418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-87120e2d-1882-4a72-b120-e8b4780efe7f None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:48.806917 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-87120e2d-1882-4a72-b120-e8b4780efe7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:48.811199 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:48.892860 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 221/428] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:32:48.894467 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf782557-5419-4fa9-a84a-d9c7e4d7f3e6 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:48.895005 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf782557-5419-4fa9-a84a-d9c7e4d7f3e6 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:48.895434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf782557-5419-4fa9-a84a-d9c7e4d7f3e6 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:48.897216 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 222/429] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:48.906856 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7db86-ca84-4e8d-9910-5f81ea1d9490 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:48.906856 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7db86-ca84-4e8d-9910-5f81ea1d9490 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:48.906856 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7db86-ca84-4e8d-9910-5f81ea1d9490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:48.927803 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-74f7db86-ca84-4e8d-9910-5f81ea1d9490 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:48.929044 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-74f7db86-ca84-4e8d-9910-5f81ea1d9490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:48.990620 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305cde3-a4f5-4a0f-b4f5-3cfd1eb6ba74 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:49.019768 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 223/430] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:32:49.062985 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ded899-2922-40e5-862b-474fdde95f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:49.064026 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ded899-2922-40e5-862b-474fdde95f5e None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:49.064521 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ded899-2922-40e5-862b-474fdde95f5e None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:49.065009 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ded899-2922-40e5-862b-474fdde95f5e None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:49.066304 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ded899-2922-40e5-862b-474fdde95f5e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:49.066880 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ded899-2922-40e5-862b-474fdde95f5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:49.069577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ded899-2922-40e5-862b-474fdde95f5e None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:49.090985 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 208/431] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:32:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:49.095198 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7db86-ca84-4e8d-9910-5f81ea1d9490 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:49.179506 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:49.219695 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7db86-ca84-4e8d-9910-5f81ea1d9490 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:49.259348 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa9922-43ef-4283-b0cc-1116feada1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:49.260326 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa9922-43ef-4283-b0cc-1116feada1db None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:49.260843 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa9922-43ef-4283-b0cc-1116feada1db None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:49.261321 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa9922-43ef-4283-b0cc-1116feada1db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:49.299374 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ded899-2922-40e5-862b-474fdde95f5e None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:49.338686 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaa9922-43ef-4283-b0cc-1116feada1db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:49.340699 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaa9922-43ef-4283-b0cc-1116feada1db None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:49.368570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:49.435631 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ded899-2922-40e5-862b-474fdde95f5e None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:49.440670 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:49.485980 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 224/432] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:32:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 22 02:32:49.486961 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa9922-43ef-4283-b0cc-1116feada1db None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:49.501755 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b03d951-564e-4f4b-ab66-546a48d1f98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:49.502963 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03d951-564e-4f4b-ab66-546a48d1f98a None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:49.503494 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03d951-564e-4f4b-ab66-546a48d1f98a None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:49.504010 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03d951-564e-4f4b-ab66-546a48d1f98a None admin] PATH_INFO: `/v3/roles/member` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:49.525246 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b03d951-564e-4f4b-ab66-546a48d1f98a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:49.536829 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-6b03d951-564e-4f4b-ab66-546a48d1f98a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 02:32:49.536829 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:49.536829 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:49.536829 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:49.536829 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:49.536829 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:49.536829 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:49.536829 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:49.536829 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:49.536829 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:49.536829 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:49.536829 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:49.536829 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:49.536829 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:49.536829 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:49.536829 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:49.537842 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:49.537842 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:49.537842 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:49.537842 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:49.537842 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:49.537842 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:49.537842 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:49.537842 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:49.537842 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:49.537842 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:49.537842 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:49.537842 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:49.537842 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:49.537842 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:49.537842 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:49.537842 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:49.538647 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:49.538647 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:49.538647 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:49.538647 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:49.538647 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:49.538647 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:49.538647 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:49.538647 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:49.538647 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:49.538647 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:49.538647 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 02:32:49.538647 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:32:49.543419 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 209/433] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:32:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:49.544590 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b03d951-564e-4f4b-ab66-546a48d1f98a None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:49.630044 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa9922-43ef-4283-b0cc-1116feada1db None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:49.695503 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d3e05-c219-445e-b0fd-5de85878cd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:49.697547 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d3e05-c219-445e-b0fd-5de85878cd66 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:49.698351 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d3e05-c219-445e-b0fd-5de85878cd66 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:49.698910 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d3e05-c219-445e-b0fd-5de85878cd66 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:49.703012 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b03d951-564e-4f4b-ab66-546a48d1f98a None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:49.715761 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:49.719091 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607d3e05-c219-445e-b0fd-5de85878cd66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:49.727696 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-607d3e05-c219-445e-b0fd-5de85878cd66 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 02:32:49.727696 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:49.727696 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:49.727696 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:49.727696 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:49.727696 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:49.727696 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:49.727696 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:49.727696 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:49.727696 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:49.727696 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:49.727696 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:49.727696 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:49.727696 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:49.727696 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:49.727696 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:49.728693 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:49.728693 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:49.728693 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:49.728693 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:49.728693 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:49.728693 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:49.728693 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:49.728693 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:49.728693 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:49.728693 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:49.728693 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:49.728693 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:49.728693 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:49.728693 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:49.728693 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:49.728693 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:49.729516 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:49.729516 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:49.729516 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:49.729516 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:49.729516 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:49.729516 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:49.729516 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:49.729516 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:49.729516 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:49.729516 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:49.729516 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 02:32:49.729516 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:32:49.729516 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 225/434] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:32:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:49.730701 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d3e05-c219-445e-b0fd-5de85878cd66 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:49.772143 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da3556-4a00-4f22-8bf5-caaadce9d57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:49.773158 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da3556-4a00-4f22-8bf5-caaadce9d57f None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:49.773582 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da3556-4a00-4f22-8bf5-caaadce9d57f None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:49.774351 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da3556-4a00-4f22-8bf5-caaadce9d57f None admin] PATH_INFO: `/v3/projects/d861cd3fc0a047d6bf06a6250256ee77` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:49.776388 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da3556-4a00-4f22-8bf5-caaadce9d57f None admin] RBAC: Authorizing `identity:get_project(project_id=d861cd3fc0a047d6bf06a6250256ee77)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:49.779023 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da3556-4a00-4f22-8bf5-caaadce9d57f None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:49.781851 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 210/435] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:32:49 2022] GET /identity/v3/projects/d861cd3fc0a047d6bf06a6250256ee77 => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:49.783082 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da3556-4a00-4f22-8bf5-caaadce9d57f None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:49.790658 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:49.917011 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d3e05-c219-445e-b0fd-5de85878cd66 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:49.954902 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da3556-4a00-4f22-8bf5-caaadce9d57f None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:49.976274 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb78f3f-347a-4184-8c34-50f3effcc165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:49.977770 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb78f3f-347a-4184-8c34-50f3effcc165 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:49.978246 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb78f3f-347a-4184-8c34-50f3effcc165 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:49.978718 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb78f3f-347a-4184-8c34-50f3effcc165 None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:49.979969 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb78f3f-347a-4184-8c34-50f3effcc165 None admin] RBAC: Adding query filter params (name=member) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:49.980504 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb78f3f-347a-4184-8c34-50f3effcc165 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:49.982950 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb78f3f-347a-4184-8c34-50f3effcc165 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:49.997726 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 226/436] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Sep 22 02:32:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:49.999114 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb78f3f-347a-4184-8c34-50f3effcc165 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:50.037044 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf7c141-da58-469d-9ed0-d1c218bf5a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:50.038063 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7c141-da58-469d-9ed0-d1c218bf5a80 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:50.038534 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7c141-da58-469d-9ed0-d1c218bf5a80 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:50.039143 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7c141-da58-469d-9ed0-d1c218bf5a80 None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:50.040871 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf7c141-da58-469d-9ed0-d1c218bf5a80 None admin] RBAC: Adding query filter params (name=member) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:50.041391 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf7c141-da58-469d-9ed0-d1c218bf5a80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:50.043938 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf7c141-da58-469d-9ed0-d1c218bf5a80 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:50.065186 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 211/437] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Sep 22 02:32:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:50.065186 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf7c141-da58-469d-9ed0-d1c218bf5a80 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:50.109714 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:50.150998 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:50.163346 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb78f3f-347a-4184-8c34-50f3effcc165 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:50.202105 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf7c141-da58-469d-9ed0-d1c218bf5a80 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:50.203895 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb3c0c-f6b3-4abc-bab0-48d6eb5a2a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:50.205034 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb3c0c-f6b3-4abc-bab0-48d6eb5a2a3a None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:50.205442 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb3c0c-f6b3-4abc-bab0-48d6eb5a2a3a None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:50.205936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb3c0c-f6b3-4abc-bab0-48d6eb5a2a3a None admin] PATH_INFO: `/v3/groups/2c48871136ff46089e386a696762e416` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:50.207921 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb3c0c-f6b3-4abc-bab0-48d6eb5a2a3a None admin] RBAC: Authorizing `identity:get_group(group_id=2c48871136ff46089e386a696762e416)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:50.210615 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb3c0c-f6b3-4abc-bab0-48d6eb5a2a3a None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:50.220008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 227/438] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Sep 22 02:32:49 2022] GET /identity/v3/groups/2c48871136ff46089e386a696762e416 => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:50.220008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb3c0c-f6b3-4abc-bab0-48d6eb5a2a3a None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:50.242291 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce39c14-3e1a-4ce1-8e4d-c476278b7b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:50.242994 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce39c14-3e1a-4ce1-8e4d-c476278b7b48 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:50.243278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce39c14-3e1a-4ce1-8e4d-c476278b7b48 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:50.243550 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce39c14-3e1a-4ce1-8e4d-c476278b7b48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:50.335183 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce39c14-3e1a-4ce1-8e4d-c476278b7b48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:50.337526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce39c14-3e1a-4ce1-8e4d-c476278b7b48 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:50.361901 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb3c0c-f6b3-4abc-bab0-48d6eb5a2a3a None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:50.380910 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:50.411392 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf9b57-5da8-4f96-957e-ca43342049e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:50.412144 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf9b57-5da8-4f96-957e-ca43342049e0 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:50.412535 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf9b57-5da8-4f96-957e-ca43342049e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:50.412930 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf9b57-5da8-4f96-957e-ca43342049e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:50.449985 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:50.468662 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 212/439] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:32:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 02:32:50.469878 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8006b-341a-45b3-a887-f16f0c533b7a None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:50.470207 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8006b-341a-45b3-a887-f16f0c533b7a None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:50.470468 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8006b-341a-45b3-a887-f16f0c533b7a None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:50.474883 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 213/440] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:50.474883 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f8006b-341a-45b3-a887-f16f0c533b7a None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:50.503960 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf9b57-5da8-4f96-957e-ca43342049e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:50.505477 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf9b57-5da8-4f96-957e-ca43342049e0 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:50.604523 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f8006b-341a-45b3-a887-f16f0c533b7a None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:50.651895 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 228/441] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:32:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 02:32:50.653319 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf9b57-5da8-4f96-957e-ca43342049e0 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:50.658880 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496f64f-83eb-4840-ae44-c6d7ce5fc2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:50.659682 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496f64f-83eb-4840-ae44-c6d7ce5fc2d1 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:50.660060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496f64f-83eb-4840-ae44-c6d7ce5fc2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:50.660427 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496f64f-83eb-4840-ae44-c6d7ce5fc2d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:50.661289 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0496f64f-83eb-4840-ae44-c6d7ce5fc2d1 None admin] RBAC: Adding query filter params (group.id=2c48871136ff46089e386a696762e416, role.id=adec3004b56243fa923bd9f0082a2728, scope.project.id=d861cd3fc0a047d6bf06a6250256ee77) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:50.661693 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0496f64f-83eb-4840-ae44-c6d7ce5fc2d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:50.663372 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0496f64f-83eb-4840-ae44-c6d7ce5fc2d1 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:50.667338 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:50.675083 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 214/442] 10.222.0.20 () {56 vars in 1376 bytes} [Thu Sep 22 02:32:50 2022] GET /identity/v3/role_assignments?group.id=2c48871136ff46089e386a696762e416&scope.project.id=d861cd3fc0a047d6bf06a6250256ee77&role.id=adec3004b56243fa923bd9f0082a2728 => generated 611 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:50.677320 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31bd6b-2fa2-4f5b-9893-e7e86dd1432c None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:50.677780 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31bd6b-2fa2-4f5b-9893-e7e86dd1432c None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:50.678220 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31bd6b-2fa2-4f5b-9893-e7e86dd1432c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:50.707932 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-4f31bd6b-2fa2-4f5b-9893-e7e86dd1432c None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:50.709818 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-4f31bd6b-2fa2-4f5b-9893-e7e86dd1432c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:50.737064 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:50.797924 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf9b57-5da8-4f96-957e-ca43342049e0 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:50.827052 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 215/443] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:32:50.829978 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f31bd6b-2fa2-4f5b-9893-e7e86dd1432c None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:50.856290 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a0c76-569e-4c19-8ec8-db29f7781773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:50.857387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a0c76-569e-4c19-8ec8-db29f7781773 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:50.857855 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a0c76-569e-4c19-8ec8-db29f7781773 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:50.858291 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a0c76-569e-4c19-8ec8-db29f7781773 None admin] PATH_INFO: `/v3/projects/d861cd3fc0a047d6bf06a6250256ee77` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:50.860553 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6a0c76-569e-4c19-8ec8-db29f7781773 None admin] RBAC: Authorizing `identity:get_project(project_id=d861cd3fc0a047d6bf06a6250256ee77)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:50.863258 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6a0c76-569e-4c19-8ec8-db29f7781773 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:50.866647 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 229/444] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:32:50 2022] GET /identity/v3/projects/d861cd3fc0a047d6bf06a6250256ee77 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:50.875118 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a0c76-569e-4c19-8ec8-db29f7781773 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:51.057703 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f31bd6b-2fa2-4f5b-9893-e7e86dd1432c None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:51.082927 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a0c76-569e-4c19-8ec8-db29f7781773 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:51.120710 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:51.129156 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:51.135454 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5696cf-5c47-479f-842d-ae38f5f4054b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:51.136856 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5696cf-5c47-479f-842d-ae38f5f4054b None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:51.137419 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5696cf-5c47-479f-842d-ae38f5f4054b None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:51.137885 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5696cf-5c47-479f-842d-ae38f5f4054b None admin] PATH_INFO: `/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:51.140093 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5696cf-5c47-479f-842d-ae38f5f4054b None admin] RBAC: Authorizing `identity:get_project(project_id=6a5cfaff6dc7495f9a0648c76c9f6eb7)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:51.142597 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5696cf-5c47-479f-842d-ae38f5f4054b None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:51.142806 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5fd4e-c2b2-480a-92c7-26f9c4b58e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:51.143671 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5fd4e-c2b2-480a-92c7-26f9c4b58e15 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:51.144082 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5fd4e-c2b2-480a-92c7-26f9c4b58e15 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:51.144488 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5fd4e-c2b2-480a-92c7-26f9c4b58e15 None admin] PATH_INFO: `/v3/groups/2c48871136ff46089e386a696762e416` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:51.153934 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 216/445] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:32:50 2022] GET /identity/v3/projects/6a5cfaff6dc7495f9a0648c76c9f6eb7 => generated 298 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:51.154181 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e5fd4e-c2b2-480a-92c7-26f9c4b58e15 None admin] RBAC: Authorizing `identity:get_group(group_id=2c48871136ff46089e386a696762e416)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:51.154181 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e5fd4e-c2b2-480a-92c7-26f9c4b58e15 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:51.154181 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 230/446] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Sep 22 02:32:50 2022] GET /identity/v3/groups/2c48871136ff46089e386a696762e416 => generated 226 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:51.155660 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5696cf-5c47-479f-842d-ae38f5f4054b None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:51.160604 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5fd4e-c2b2-480a-92c7-26f9c4b58e15 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:51.312682 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5fd4e-c2b2-480a-92c7-26f9c4b58e15 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:51.331998 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5696cf-5c47-479f-842d-ae38f5f4054b None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:51.353977 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515675e3-6ffa-4dd6-ac37-244a0873e0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:51.354739 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515675e3-6ffa-4dd6-ac37-244a0873e0c6 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:51.355124 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515675e3-6ffa-4dd6-ac37-244a0873e0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:51.355460 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515675e3-6ffa-4dd6-ac37-244a0873e0c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:51.356381 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515675e3-6ffa-4dd6-ac37-244a0873e0c6 None admin] RBAC: Adding query filter params (group.id=2c48871136ff46089e386a696762e416, role.id=83efac3b92eb44e783f4924c9d9ee842, scope.project.id=d861cd3fc0a047d6bf06a6250256ee77) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:51.356900 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515675e3-6ffa-4dd6-ac37-244a0873e0c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:51.359491 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515675e3-6ffa-4dd6-ac37-244a0873e0c6 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:51.367905 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 231/447] 10.222.0.20 () {56 vars in 1376 bytes} [Thu Sep 22 02:32:51 2022] GET /identity/v3/role_assignments?group.id=2c48871136ff46089e386a696762e416&scope.project.id=d861cd3fc0a047d6bf06a6250256ee77&role.id=83efac3b92eb44e783f4924c9d9ee842 => generated 611 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:51.369316 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515675e3-6ffa-4dd6-ac37-244a0873e0c6 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:51.379848 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ce39a-23b0-448b-8c46-f7eca0e8f01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:51.380825 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ce39a-23b0-448b-8c46-f7eca0e8f01f None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:51.381260 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ce39a-23b0-448b-8c46-f7eca0e8f01f None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:51.381685 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ce39a-23b0-448b-8c46-f7eca0e8f01f None admin] PATH_INFO: `/v3/groups/2c48871136ff46089e386a696762e416` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:51.384169 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9ce39a-23b0-448b-8c46-f7eca0e8f01f None admin] RBAC: Authorizing `identity:get_group(group_id=2c48871136ff46089e386a696762e416)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:51.386629 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9ce39a-23b0-448b-8c46-f7eca0e8f01f None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:51.395360 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 217/448] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Sep 22 02:32:51 2022] GET /identity/v3/groups/2c48871136ff46089e386a696762e416 => generated 226 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:51.399322 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ce39a-23b0-448b-8c46-f7eca0e8f01f None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:51.475281 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:51.529487 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:51.544159 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515675e3-6ffa-4dd6-ac37-244a0873e0c6 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:51.581731 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ce39a-23b0-448b-8c46-f7eca0e8f01f None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:51.610723 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20c3df-ab57-4f2d-9209-fa54c7d2d855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:51.612034 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20c3df-ab57-4f2d-9209-fa54c7d2d855 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:51.612558 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20c3df-ab57-4f2d-9209-fa54c7d2d855 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:51.613021 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20c3df-ab57-4f2d-9209-fa54c7d2d855 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:51.633563 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20c3df-ab57-4f2d-9209-fa54c7d2d855 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:51.644055 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-9b20c3df-ab57-4f2d-9209-fa54c7d2d855 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:32:51.644055 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:51.644055 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:51.644055 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:51.644055 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:51.644055 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:51.644055 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:51.644055 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:51.644055 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:51.644055 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:51.644055 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:51.644055 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:51.644055 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:51.644055 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:51.644055 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:51.644055 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:51.645280 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:51.645280 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:51.645280 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:51.645280 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:51.645280 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:51.645280 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:51.645280 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:51.645280 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:51.645280 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:51.645280 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:51.645280 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:51.645280 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:51.645280 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:51.645280 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:51.645280 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:51.645280 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:51.648422 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:51.648422 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:51.648422 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:51.648422 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:51.648422 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:51.648422 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:51.648422 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:51.648422 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:51.648422 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:51.648422 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:51.648422 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:32:51.648422 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:32:51.648422 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 232/449] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Sep 22 02:32:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:51.650050 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b58d3d-950e-4949-b72d-81aecf1bc0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:51.650050 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b58d3d-950e-4949-b72d-81aecf1bc0ca None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:51.650050 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b58d3d-950e-4949-b72d-81aecf1bc0ca None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:51.650050 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b58d3d-950e-4949-b72d-81aecf1bc0ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:51.651772 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b58d3d-950e-4949-b72d-81aecf1bc0ca None admin] RBAC: Adding query filter params (group.id=2c48871136ff46089e386a696762e416, role.id=83efac3b92eb44e783f4924c9d9ee842, scope.project.id=6a5cfaff6dc7495f9a0648c76c9f6eb7) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:51.651772 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b58d3d-950e-4949-b72d-81aecf1bc0ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:51.652796 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20c3df-ab57-4f2d-9209-fa54c7d2d855 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:51.654296 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b58d3d-950e-4949-b72d-81aecf1bc0ca None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:51.664384 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 218/450] 10.222.0.20 () {56 vars in 1376 bytes} [Thu Sep 22 02:32:51 2022] GET /identity/v3/role_assignments?group.id=2c48871136ff46089e386a696762e416&scope.project.id=6a5cfaff6dc7495f9a0648c76c9f6eb7&role.id=83efac3b92eb44e783f4924c9d9ee842 => generated 611 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:32:51.768038 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20c3df-ab57-4f2d-9209-fa54c7d2d855 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:51.788228 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:51.808580 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d286bba2-6f1f-4fef-beaa-e56216697113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:51.809208 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d286bba2-6f1f-4fef-beaa-e56216697113 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:51.809486 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d286bba2-6f1f-4fef-beaa-e56216697113 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:51.809758 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d286bba2-6f1f-4fef-beaa-e56216697113 None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:51.810439 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d286bba2-6f1f-4fef-beaa-e56216697113 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:51.810744 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d286bba2-6f1f-4fef-beaa-e56216697113 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:51.812222 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d286bba2-6f1f-4fef-beaa-e56216697113 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:51.822113 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 233/451] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:32:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:32:51.828741 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b58d3d-950e-4949-b72d-81aecf1bc0ca None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:51.959231 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b58d3d-950e-4949-b72d-81aecf1bc0ca None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:52.013983 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:52.025690 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974f404-8fe3-4128-a4cc-f24390b107ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:52.026761 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974f404-8fe3-4128-a4cc-f24390b107ee None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:52.027290 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974f404-8fe3-4128-a4cc-f24390b107ee None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:52.027778 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974f404-8fe3-4128-a4cc-f24390b107ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:52.134743 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9974f404-8fe3-4128-a4cc-f24390b107ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:52.136778 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9974f404-8fe3-4128-a4cc-f24390b107ee None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:52.287583 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 219/452] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:32:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 02:32:52.295180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d286bba2-6f1f-4fef-beaa-e56216697113 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:52.428402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d286bba2-6f1f-4fef-beaa-e56216697113 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:52.473452 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5221c-f1c3-421d-859e-836021d2c16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:52.474472 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5221c-f1c3-421d-859e-836021d2c16a None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:52.475022 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5221c-f1c3-421d-859e-836021d2c16a None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:52.475486 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5221c-f1c3-421d-859e-836021d2c16a None admin] PATH_INFO: `/v3/projects/f92fe1b089244bf4b21f4f98339aa81d` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:52.476933 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5221c-f1c3-421d-859e-836021d2c16a None admin] RBAC: Authorizing `identity:get_project(project_id=f92fe1b089244bf4b21f4f98339aa81d)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:52.479386 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5221c-f1c3-421d-859e-836021d2c16a None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:52.481806 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 234/453] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:32:52 2022] GET /identity/v3/projects/f92fe1b089244bf4b21f4f98339aa81d => generated 344 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:52.489598 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974f404-8fe3-4128-a4cc-f24390b107ee None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:52.534439 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:52.631834 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974f404-8fe3-4128-a4cc-f24390b107ee None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:52.679949 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a05f88-ef27-41db-8b3a-efe0f704c2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:52.680975 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a05f88-ef27-41db-8b3a-efe0f704c2a1 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:52.681325 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a05f88-ef27-41db-8b3a-efe0f704c2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:52.681702 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a05f88-ef27-41db-8b3a-efe0f704c2a1 None admin] PATH_INFO: `/v3/groups/0c4aa1bea31a44159a2b8a7cf69fa0a6` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:52.683332 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a05f88-ef27-41db-8b3a-efe0f704c2a1 None admin] RBAC: Authorizing `identity:get_group(group_id=0c4aa1bea31a44159a2b8a7cf69fa0a6)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:52.685004 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a05f88-ef27-41db-8b3a-efe0f704c2a1 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:52.688474 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 220/454] 10.222.0.20 () {56 vars in 1180 bytes} [Thu Sep 22 02:32:52 2022] GET /identity/v3/groups/0c4aa1bea31a44159a2b8a7cf69fa0a6 => generated 229 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:52.693836 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5221c-f1c3-421d-859e-836021d2c16a None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:52.753172 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:52.806159 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5221c-f1c3-421d-859e-836021d2c16a None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:52.839559 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51391d73-4198-4e3a-a097-2c0d46bfc9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:52.840161 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51391d73-4198-4e3a-a097-2c0d46bfc9a3 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:52.840444 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51391d73-4198-4e3a-a097-2c0d46bfc9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:52.840718 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51391d73-4198-4e3a-a097-2c0d46bfc9a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:52.841442 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51391d73-4198-4e3a-a097-2c0d46bfc9a3 None admin] RBAC: Adding query filter params (group.id=0c4aa1bea31a44159a2b8a7cf69fa0a6, role.id=31a687548f55493b8ea666fe7e350303, scope.project.id=f92fe1b089244bf4b21f4f98339aa81d) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:52.841750 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51391d73-4198-4e3a-a097-2c0d46bfc9a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:52.843199 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51391d73-4198-4e3a-a097-2c0d46bfc9a3 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:52.848926 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 235/455] 10.222.0.20 () {56 vars in 1376 bytes} [Thu Sep 22 02:32:52 2022] GET /identity/v3/role_assignments?group.id=0c4aa1bea31a44159a2b8a7cf69fa0a6&scope.project.id=f92fe1b089244bf4b21f4f98339aa81d&role.id=31a687548f55493b8ea666fe7e350303 => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:55.605128 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fcf69-c097-49c6-8b66-6bd049668f7a None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:55.606117 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fcf69-c097-49c6-8b66-6bd049668f7a None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:55.606566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fcf69-c097-49c6-8b66-6bd049668f7a None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:55.608260 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 221/456] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:55.614117 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d59b35-e251-40d8-ad62-51005f71e8e3 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:55.615029 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d59b35-e251-40d8-ad62-51005f71e8e3 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:55.615457 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d59b35-e251-40d8-ad62-51005f71e8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:55.637742 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-c0d59b35-e251-40d8-ad62-51005f71e8e3 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:55.639266 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-c0d59b35-e251-40d8-ad62-51005f71e8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:55.729455 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 236/457] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:32:56.041052 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931fcf69-c097-49c6-8b66-6bd049668f7a None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:56.246081 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:56.256613 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931fcf69-c097-49c6-8b66-6bd049668f7a None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:56.310497 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225d859c-46d9-4543-b269-8bcaf08eef68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:56.311141 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d859c-46d9-4543-b269-8bcaf08eef68 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:56.311429 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d859c-46d9-4543-b269-8bcaf08eef68 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:56.311699 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d859c-46d9-4543-b269-8bcaf08eef68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:56.324974 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225d859c-46d9-4543-b269-8bcaf08eef68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:56.326432 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225d859c-46d9-4543-b269-8bcaf08eef68 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:56.343837 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-225d859c-46d9-4543-b269-8bcaf08eef68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:32:56.343837 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:56.343837 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:56.343837 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:56.343837 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:56.343837 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:56.343837 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:56.343837 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:56.343837 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:56.343837 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:56.343837 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:56.343837 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:56.343837 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:32:56.343837 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:32:56.343837 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:32:56.343837 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:32:56.344812 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:56.344812 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:56.344812 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:56.344812 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:56.344812 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:56.344812 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:56.344812 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:56.344812 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:56.344812 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:56.344812 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:56.344812 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:56.344812 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:56.344812 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:56.344812 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:56.344812 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:56.344812 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:56.344812 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:32:56.345670 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:32:56.345670 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:32:56.345670 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:32:56.347945 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 222/458] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:32:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:56.356901 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d59b35-e251-40d8-ad62-51005f71e8e3 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:56.431346 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:56.519043 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d59b35-e251-40d8-ad62-51005f71e8e3 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:56.575622 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd0abe-18fc-423b-8d4b-73c04a97070b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:56.576274 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd0abe-18fc-423b-8d4b-73c04a97070b None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:56.576561 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd0abe-18fc-423b-8d4b-73c04a97070b None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:56.576843 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd0abe-18fc-423b-8d4b-73c04a97070b None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:56.577512 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd0abe-18fc-423b-8d4b-73c04a97070b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:56.577819 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd0abe-18fc-423b-8d4b-73c04a97070b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:56.579244 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd0abe-18fc-423b-8d4b-73c04a97070b None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:56.591948 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 237/459] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:32:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:56.598442 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225d859c-46d9-4543-b269-8bcaf08eef68 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:56.684633 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea49134-40ba-4a57-a494-933609cf2581 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:56.685191 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea49134-40ba-4a57-a494-933609cf2581 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:56.685191 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea49134-40ba-4a57-a494-933609cf2581 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:56.686520 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 238/460] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:32:56.697734 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f0b97-6782-4202-bcd1-ece12c8af3f5 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:56.698399 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f0b97-6782-4202-bcd1-ece12c8af3f5 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:56.698928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f0b97-6782-4202-bcd1-ece12c8af3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:56.720630 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-d36f0b97-6782-4202-bcd1-ece12c8af3f5 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:56.722196 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-d36f0b97-6782-4202-bcd1-ece12c8af3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:56.808063 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 239/461] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 02:32:56.852986 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225d859c-46d9-4543-b269-8bcaf08eef68 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:56.894413 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1423dec2-798b-48a6-ba82-d45806c09097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:56.895445 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1423dec2-798b-48a6-ba82-d45806c09097 None admin] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:56.895948 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1423dec2-798b-48a6-ba82-d45806c09097 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:56.896421 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1423dec2-798b-48a6-ba82-d45806c09097 None admin] PATH_INFO: `/v3/users` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:56.897273 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1423dec2-798b-48a6-ba82-d45806c09097 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:56.899778 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1423dec2-798b-48a6-ba82-d45806c09097 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:56.930497 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6fc9c-d4f2-441d-bd8d-24c74de63ac0 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:56.932498 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 223/462] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Sep 22 02:32:56 2022] POST /identity/v3/users => generated 246 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:32:56.932932 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6fc9c-d4f2-441d-bd8d-24c74de63ac0 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:56.933431 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6fc9c-d4f2-441d-bd8d-24c74de63ac0 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:56.941890 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 240/463] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:56.943947 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd16b51-3dc5-4b48-b84a-8b76ab4542a1 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:56.944488 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd16b51-3dc5-4b48-b84a-8b76ab4542a1 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:56.944947 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd16b51-3dc5-4b48-b84a-8b76ab4542a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:56.960049 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:56.966671 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-0cd16b51-3dc5-4b48-b84a-8b76ab4542a1 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:56.968582 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-0cd16b51-3dc5-4b48-b84a-8b76ab4542a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:57.049053 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 224/464] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:32:57.219439 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f9d0e-f9f4-4d19-809f-8122afb1e234 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:57.219687 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f9d0e-f9f4-4d19-809f-8122afb1e234 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:57.219859 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f9d0e-f9f4-4d19-809f-8122afb1e234 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:57.221552 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 241/465] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:57.233652 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea4fcc-8767-4cd0-a166-87aa5eee5dbf None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:57.235708 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea4fcc-8767-4cd0-a166-87aa5eee5dbf None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:57.236255 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea4fcc-8767-4cd0-a166-87aa5eee5dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:57.242086 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f9d0e-f9f4-4d19-809f-8122afb1e234 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:57.269468 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-abea4fcc-8767-4cd0-a166-87aa5eee5dbf None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:57.271545 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-abea4fcc-8767-4cd0-a166-87aa5eee5dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:57.307481 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:57.359055 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 225/466] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:32:57.428402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f9d0e-f9f4-4d19-809f-8122afb1e234 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:57.472498 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98246757-657d-4262-ae22-3b51f9690276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:57.473074 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98246757-657d-4262-ae22-3b51f9690276 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:57.473354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98246757-657d-4262-ae22-3b51f9690276 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:57.473625 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98246757-657d-4262-ae22-3b51f9690276 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:57.478402 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea4fcc-8767-4cd0-a166-87aa5eee5dbf None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:57.489732 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98246757-657d-4262-ae22-3b51f9690276 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:57.491264 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98246757-657d-4262-ae22-3b51f9690276 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:57.514637 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-98246757-657d-4262-ae22-3b51f9690276 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:32:57.514637 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:57.514637 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:57.514637 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:57.514637 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:57.514637 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:57.514637 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:57.514637 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:57.514637 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:57.514637 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:57.514637 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:57.514637 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:57.514637 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:32:57.514637 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:32:57.514637 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:32:57.514637 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:32:57.515653 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:57.515653 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:57.515653 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:57.515653 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:57.515653 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:57.515653 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:57.515653 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:57.515653 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:57.515653 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:57.515653 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:57.515653 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:57.515653 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:57.515653 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:57.515653 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:57.515653 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:57.515653 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:57.515653 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:32:57.516733 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:32:57.516733 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:32:57.516733 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:32:57.516733 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 242/467] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:32:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:57.522158 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98246757-657d-4262-ae22-3b51f9690276 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:57.633829 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:57.678565 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea4fcc-8767-4cd0-a166-87aa5eee5dbf None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:57.715715 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:57.730347 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed00511-148f-4e01-8f6d-29cafb8c2e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:57.730764 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed00511-148f-4e01-8f6d-29cafb8c2e9f None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:57.731066 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed00511-148f-4e01-8f6d-29cafb8c2e9f None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:57.731331 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed00511-148f-4e01-8f6d-29cafb8c2e9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:57.740845 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98246757-657d-4262-ae22-3b51f9690276 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:57.743291 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed00511-148f-4e01-8f6d-29cafb8c2e9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:57.744776 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed00511-148f-4e01-8f6d-29cafb8c2e9f None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:57.761137 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-2ed00511-148f-4e01-8f6d-29cafb8c2e9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:32:57.761137 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:57.761137 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:57.761137 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:57.761137 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:57.761137 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:57.761137 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:57.761137 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:57.761137 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:57.761137 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:57.761137 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:57.761137 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:57.761137 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:32:57.761137 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:32:57.761137 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:32:57.761137 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:32:57.763395 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:57.763395 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:57.763395 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:57.763395 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:57.763395 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:57.763395 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:57.763395 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:57.763395 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:57.763395 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:57.763395 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:57.763395 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:57.763395 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:57.763395 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:57.763395 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:57.763395 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:57.763395 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:57.763395 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:32:57.764972 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:32:57.764972 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:32:57.764972 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:32:57.764972 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 226/468] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:32:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:32:57.764972 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed00511-148f-4e01-8f6d-29cafb8c2e9f None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:57.814237 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf5028-0322-42c4-aabb-b9cb884d7839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:57.815804 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf5028-0322-42c4-aabb-b9cb884d7839 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:57.816789 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf5028-0322-42c4-aabb-b9cb884d7839 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:57.817691 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf5028-0322-42c4-aabb-b9cb884d7839 None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:57.819270 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf5028-0322-42c4-aabb-b9cb884d7839 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:57.820347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf5028-0322-42c4-aabb-b9cb884d7839 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:57.823186 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf5028-0322-42c4-aabb-b9cb884d7839 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:57.839146 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 243/469] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:32:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:57.840463 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf5028-0322-42c4-aabb-b9cb884d7839 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:57.883547 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed00511-148f-4e01-8f6d-29cafb8c2e9f None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:57.922464 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:57.960476 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b7afe3-a570-42e2-ac72-4fa5736d6918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:57.961646 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7afe3-a570-42e2-ac72-4fa5736d6918 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:57.962089 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7afe3-a570-42e2-ac72-4fa5736d6918 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:57.962517 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7afe3-a570-42e2-ac72-4fa5736d6918 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:57.982606 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf5028-0322-42c4-aabb-b9cb884d7839 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:58.000225 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b7afe3-a570-42e2-ac72-4fa5736d6918 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:58.003205 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b7afe3-a570-42e2-ac72-4fa5736d6918 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:58.038362 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f12faa2-7766-4a30-bb04-c510220f5f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:58.039496 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12faa2-7766-4a30-bb04-c510220f5f31 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:58.039997 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12faa2-7766-4a30-bb04-c510220f5f31 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:58.040490 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12faa2-7766-4a30-bb04-c510220f5f31 None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:58.042243 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f12faa2-7766-4a30-bb04-c510220f5f31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:58.043173 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f12faa2-7766-4a30-bb04-c510220f5f31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:58.044844 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f12faa2-7766-4a30-bb04-c510220f5f31 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:58.051448 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-d1b7afe3-a570-42e2-ac72-4fa5736d6918 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:32:58.051448 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:58.051448 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:58.051448 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:58.051448 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:58.051448 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:58.051448 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:58.051448 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:58.051448 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:58.051448 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:58.051448 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:58.051448 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:58.051448 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:32:58.051448 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:32:58.051448 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:32:58.051448 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:32:58.052422 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:58.052422 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:58.052422 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:58.052422 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:58.052422 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:58.052422 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:58.052422 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:58.052422 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:58.052422 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:58.052422 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:58.052422 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:58.052422 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:58.052422 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:58.052422 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:58.052422 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:58.052422 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:58.052422 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:32:58.053303 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:32:58.053303 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:32:58.053303 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:32:58.063350 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 244/470] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:32:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:58.063716 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 227/471] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:32:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:58.064731 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b7afe3-a570-42e2-ac72-4fa5736d6918 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:58.065034 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f12faa2-7766-4a30-bb04-c510220f5f31 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:58.074125 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:58.201413 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f12faa2-7766-4a30-bb04-c510220f5f31 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:58.229069 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b7afe3-a570-42e2-ac72-4fa5736d6918 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:58.243302 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319607b-64ee-4600-a606-ee8043f33251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:58.243851 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319607b-64ee-4600-a606-ee8043f33251 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:58.244135 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319607b-64ee-4600-a606-ee8043f33251 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:58.244412 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319607b-64ee-4600-a606-ee8043f33251 None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:58.245115 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319607b-64ee-4600-a606-ee8043f33251 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:32:58.245453 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319607b-64ee-4600-a606-ee8043f33251 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:58.248134 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319607b-64ee-4600-a606-ee8043f33251 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:58.262742 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 245/472] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:32:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:32:58.263643 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319607b-64ee-4600-a606-ee8043f33251 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:58.289443 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c0d557-0e0a-4bc2-8168-c0830a9a9ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:58.290325 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0d557-0e0a-4bc2-8168-c0830a9a9ed6 None admin] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:58.290607 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0d557-0e0a-4bc2-8168-c0830a9a9ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:58.290919 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0d557-0e0a-4bc2-8168-c0830a9a9ed6 None admin] PATH_INFO: `/v3/users` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:58.291588 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c0d557-0e0a-4bc2-8168-c0830a9a9ed6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:58.293042 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c0d557-0e0a-4bc2-8168-c0830a9a9ed6 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:58.320217 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 228/473] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Sep 22 02:32:58 2022] POST /identity/v3/users => generated 248 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:32:58.323593 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580bc83-13e0-4a3c-bfc3-61fb82b576a4 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:58.323889 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580bc83-13e0-4a3c-bfc3-61fb82b576a4 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:58.324169 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580bc83-13e0-4a3c-bfc3-61fb82b576a4 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:58.325611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 229/474] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:32:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:32:58.326410 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5580bc83-13e0-4a3c-bfc3-61fb82b576a4 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:58.399811 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:58.405484 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:58.504384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319607b-64ee-4600-a606-ee8043f33251 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:58.583941 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5580bc83-13e0-4a3c-bfc3-61fb82b576a4 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:58.598194 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e263c77d-1e80-444f-ab7f-0360f603394d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:58.598863 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263c77d-1e80-444f-ab7f-0360f603394d None admin] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:58.599381 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263c77d-1e80-444f-ab7f-0360f603394d None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:58.599799 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263c77d-1e80-444f-ab7f-0360f603394d None admin] PATH_INFO: `/v3/users` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:58.600872 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e263c77d-1e80-444f-ab7f-0360f603394d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:58.604147 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e263c77d-1e80-444f-ab7f-0360f603394d None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:58.649189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 246/475] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Sep 22 02:32:58 2022] POST /identity/v3/users => generated 248 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:32:58.651916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2da2b0-a3e6-4a8e-b3cb-406b75b7b32e None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:58.652586 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2da2b0-a3e6-4a8e-b3cb-406b75b7b32e None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:58.653172 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2da2b0-a3e6-4a8e-b3cb-406b75b7b32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:58.666443 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0394b75a-e9c2-43c7-9259-9b923b3c9f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:58.667370 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394b75a-e9c2-43c7-9259-9b923b3c9f9b None admin] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:58.667800 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394b75a-e9c2-43c7-9259-9b923b3c9f9b None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:58.668245 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394b75a-e9c2-43c7-9259-9b923b3c9f9b None admin] PATH_INFO: `/v3/users` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:58.669215 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0394b75a-e9c2-43c7-9259-9b923b3c9f9b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:58.671004 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0394b75a-e9c2-43c7-9259-9b923b3c9f9b None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:58.689983 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-9e2da2b0-a3e6-4a8e-b3cb-406b75b7b32e None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:32:58.693089 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-9e2da2b0-a3e6-4a8e-b3cb-406b75b7b32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:32:58.701253 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 230/476] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Sep 22 02:32:58 2022] POST /identity/v3/users => generated 249 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:32:58.815278 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 247/477] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:32:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:32:59.165470 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0394b75a-e9c2-43c7-9259-9b923b3c9f9b None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:59.357160 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:59.388032 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0394b75a-e9c2-43c7-9259-9b923b3c9f9b None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:59.444277 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b25e521-ea62-46ea-83a7-47ce3ae584f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:59.444834 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b25e521-ea62-46ea-83a7-47ce3ae584f6 None admin] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:59.445121 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b25e521-ea62-46ea-83a7-47ce3ae584f6 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:59.445969 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b25e521-ea62-46ea-83a7-47ce3ae584f6 None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:59.446669 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b25e521-ea62-46ea-83a7-47ce3ae584f6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:32:59.448119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b25e521-ea62-46ea-83a7-47ce3ae584f6 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:32:59.460693 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.sql.core [None req-6b25e521-ea62-46ea-83a7-47ce3ae584f6 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 22 02:32:59.460693 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 22 02:32:59.460693 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [parameters: {'id': '68cc10f07de045419660641426b00677', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 22 02:32:59.460693 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88065) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 22 02:32:59.482213 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-6b25e521-ea62-46ea-83a7-47ce3ae584f6 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 22 02:32:59.482213 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:59.482213 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 22 02:32:59.482213 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 22 02:32:59.482213 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 22 02:32:59.482213 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 22 02:32:59.482213 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 22 02:32:59.482213 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application result = self._query(query) Sep 22 02:32:59.482213 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 22 02:32:59.482213 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application conn.query(q) Sep 22 02:32:59.482213 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 22 02:32:59.482213 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 22 02:32:59.482213 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 22 02:32:59.482213 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application result.read() Sep 22 02:32:59.482213 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 22 02:32:59.483208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 22 02:32:59.483208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 22 02:32:59.483208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 22 02:32:59.483208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 22 02:32:59.483208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 22 02:32:59.483208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 22 02:32:59.483208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 22 02:32:59.483208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 22 02:32:59.483208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:32:59.483208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 22 02:32:59.483208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:32:59.483208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:59.483208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 22 02:32:59.483208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 22 02:32:59.483208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 22 02:32:59.483208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return ref.to_dict() Sep 22 02:32:59.483208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 22 02:32:59.483208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application next(self.gen) Sep 22 02:32:59.484126 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 22 02:32:59.484126 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application yield resource Sep 22 02:32:59.484126 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 22 02:32:59.484126 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application next(self.gen) Sep 22 02:32:59.484126 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 22 02:32:59.484126 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self.session.rollback() Sep 22 02:32:59.484126 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 22 02:32:59.484126 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self.force_reraise() Sep 22 02:32:59.484126 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 22 02:32:59.484126 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise self.value Sep 22 02:32:59.484126 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 22 02:32:59.484126 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 22 02:32:59.484126 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 22 02:32:59.484126 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application session.commit() Sep 22 02:32:59.484126 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 22 02:32:59.484126 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 22 02:32:59.484126 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 22 02:32:59.485663 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self._prepare_impl() Sep 22 02:32:59.485663 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 22 02:32:59.485663 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self.session.flush() Sep 22 02:32:59.485663 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 22 02:32:59.485663 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self._flush(objects) Sep 22 02:32:59.485663 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 22 02:32:59.485663 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 22 02:32:59.485663 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 22 02:32:59.485663 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application compat.raise_( Sep 22 02:32:59.485663 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 22 02:32:59.485663 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception Sep 22 02:32:59.485663 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 22 02:32:59.485663 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application flush_context.execute() Sep 22 02:32:59.485663 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 22 02:32:59.485663 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rec.execute(self) Sep 22 02:32:59.485663 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 22 02:32:59.485663 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 22 02:32:59.486575 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 22 02:32:59.486575 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 22 02:32:59.486575 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 22 02:32:59.486575 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 22 02:32:59.486575 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 22 02:32:59.486575 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 22 02:32:59.486575 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 22 02:32:59.486575 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 22 02:32:59.486575 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 22 02:32:59.486575 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 22 02:32:59.486575 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 22 02:32:59.486575 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 22 02:32:59.486575 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 22 02:32:59.486575 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 22 02:32:59.486575 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 22 02:32:59.486575 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception Sep 22 02:32:59.488300 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 22 02:32:59.488300 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 22 02:32:59.488300 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 22 02:32:59.488300 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 22 02:32:59.488300 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 22 02:32:59.488300 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application result = self._query(query) Sep 22 02:32:59.488300 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 22 02:32:59.488300 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application conn.query(q) Sep 22 02:32:59.488300 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 22 02:32:59.488300 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 22 02:32:59.488300 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 22 02:32:59.488300 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application result.read() Sep 22 02:32:59.488300 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 22 02:32:59.488300 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 22 02:32:59.488300 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 22 02:32:59.488300 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 22 02:32:59.490241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 22 02:32:59.490241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 22 02:32:59.490241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 22 02:32:59.490241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 22 02:32:59.490241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 22 02:32:59.490241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 22 02:32:59.490241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [parameters: {'id': '68cc10f07de045419660641426b00677', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 22 02:32:59.490241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 22 02:32:59.490241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:32:59.490241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 02:32:59.490241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:32:59.490241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:59.490241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:59.490241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:59.490241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:59.490241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:59.492791 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:59.492791 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:59.492791 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:59.492791 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:59.492791 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:59.492791 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:59.492791 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 22 02:32:59.492791 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 22 02:32:59.492791 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:59.492791 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:59.492791 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 22 02:32:59.492791 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 22 02:32:59.492791 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 22 02:32:59.492791 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 22 02:32:59.492791 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 22 02:32:59.492791 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:32:59.496326 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 231/478] 10.222.0.20 () {60 vars in 1131 bytes} [Thu Sep 22 02:32:59 2022] POST /identity/v3/roles => generated 144 bytes in 320 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:59.496502 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2da2b0-a3e6-4a8e-b3cb-406b75b7b32e None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:59.607637 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:32:59.703253 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2da2b0-a3e6-4a8e-b3cb-406b75b7b32e None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:32:59.769342 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4cf72b-2f47-4fdd-bafc-27950c73f59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:32:59.770005 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4cf72b-2f47-4fdd-bafc-27950c73f59b None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:32:59.770292 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4cf72b-2f47-4fdd-bafc-27950c73f59b None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:32:59.770570 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4cf72b-2f47-4fdd-bafc-27950c73f59b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:32:59.787806 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4cf72b-2f47-4fdd-bafc-27950c73f59b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:32:59.793457 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-7c4cf72b-2f47-4fdd-bafc-27950c73f59b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 02:32:59.793457 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:32:59.793457 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:32:59.793457 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:32:59.793457 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:32:59.793457 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:32:59.793457 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:32:59.793457 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:32:59.793457 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:32:59.793457 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:32:59.793457 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:32:59.793457 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:32:59.793457 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:32:59.793457 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:32:59.793457 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:32:59.793457 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:32:59.795633 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:32:59.795633 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:32:59.795633 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:32:59.795633 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:32:59.795633 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:32:59.795633 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:32:59.795633 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:32:59.795633 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:32:59.795633 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:32:59.795633 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:32:59.795633 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:32:59.795633 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:32:59.795633 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:32:59.795633 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:32:59.795633 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:32:59.795633 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:32:59.797596 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:32:59.797596 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:32:59.797596 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:32:59.797596 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:32:59.797596 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:32:59.797596 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:32:59.797596 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:32:59.797596 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:32:59.797596 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:32:59.797596 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:32:59.797596 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 02:32:59.797596 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:32:59.799145 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 248/479] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Sep 22 02:32:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:32:59.801020 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b25e521-ea62-46ea-83a7-47ce3ae584f6 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:32:59.962708 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b25e521-ea62-46ea-83a7-47ce3ae584f6 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:00.016728 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf52ae-01af-42c6-ae97-31fce62c1d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:00.017755 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf52ae-01af-42c6-ae97-31fce62c1d36 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:00.018256 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf52ae-01af-42c6-ae97-31fce62c1d36 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:00.018732 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf52ae-01af-42c6-ae97-31fce62c1d36 None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:00.019648 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf52ae-01af-42c6-ae97-31fce62c1d36 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:00.020158 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf52ae-01af-42c6-ae97-31fce62c1d36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:00.021782 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf52ae-01af-42c6-ae97-31fce62c1d36 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:00.037645 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 232/480] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:32:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:00.315248 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc5462c-73e7-4682-9706-ade863c29e28 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:00.316287 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc5462c-73e7-4682-9706-ade863c29e28 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:00.316795 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc5462c-73e7-4682-9706-ade863c29e28 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:00.318728 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 249/481] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:33:00.327863 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4237fd-2ab1-4b58-bfda-5b871ce7ce77 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:00.328399 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4237fd-2ab1-4b58-bfda-5b871ce7ce77 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:00.328793 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4237fd-2ab1-4b58-bfda-5b871ce7ce77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:00.369211 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-3d4237fd-2ab1-4b58-bfda-5b871ce7ce77 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:00.371452 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-3d4237fd-2ab1-4b58-bfda-5b871ce7ce77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:00.375453 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:00.513183 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 233/482] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 02:33:01.027876 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc5462c-73e7-4682-9706-ade863c29e28 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:01.170110 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc5462c-73e7-4682-9706-ade863c29e28 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:01.189520 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:01.229470 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65185e7-2580-4285-a37f-6d472ac65960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:01.230132 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65185e7-2580-4285-a37f-6d472ac65960 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:01.230419 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65185e7-2580-4285-a37f-6d472ac65960 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:01.230698 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65185e7-2580-4285-a37f-6d472ac65960 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:01.254022 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65185e7-2580-4285-a37f-6d472ac65960 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:33:01.259970 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-a65185e7-2580-4285-a37f-6d472ac65960 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:33:01.259970 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:01.259970 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:01.259970 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:01.259970 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:01.259970 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:01.259970 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:01.259970 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:01.259970 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:01.259970 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:01.259970 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:01.259970 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:01.259970 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:33:01.259970 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:33:01.259970 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:33:01.259970 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:33:01.261607 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:33:01.261607 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:33:01.261607 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:33:01.261607 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:33:01.261607 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:01.261607 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:01.261607 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:01.261607 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:01.261607 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:01.261607 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:01.261607 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:01.261607 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:01.261607 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:01.261607 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:01.261607 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:01.261607 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:01.263262 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:01.263262 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:01.263262 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:01.263262 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:01.263262 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:33:01.263262 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:33:01.263262 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:33:01.263262 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:33:01.263262 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:33:01.263262 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:33:01.263262 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:33:01.263262 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:01.270857 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 250/483] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Sep 22 02:33:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:01.275495 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4237fd-2ab1-4b58-bfda-5b871ce7ce77 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:01.456992 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84727f4b-1a41-4464-b512-ee4166a0c304 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:01.457808 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84727f4b-1a41-4464-b512-ee4166a0c304 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:01.457808 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84727f4b-1a41-4464-b512-ee4166a0c304 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:01.460465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 251/484] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:33:01.472739 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc59b4-3f13-45e1-b0e8-bea3763100b0 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:01.473128 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc59b4-3f13-45e1-b0e8-bea3763100b0 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:01.473420 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc59b4-3f13-45e1-b0e8-bea3763100b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:01.506722 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-03fc59b4-3f13-45e1-b0e8-bea3763100b0 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:01.509797 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-03fc59b4-3f13-45e1-b0e8-bea3763100b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:01.515652 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4237fd-2ab1-4b58-bfda-5b871ce7ce77 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:01.600593 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5531b-4c09-405f-9257-4778d20cc195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:01.601867 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5531b-4c09-405f-9257-4778d20cc195 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:01.603582 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5531b-4c09-405f-9257-4778d20cc195 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:01.604588 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5531b-4c09-405f-9257-4778d20cc195 None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:01.605958 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab5531b-4c09-405f-9257-4778d20cc195 None admin] RBAC: Adding query filter params (name=service) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:01.607063 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab5531b-4c09-405f-9257-4778d20cc195 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:01.610088 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab5531b-4c09-405f-9257-4778d20cc195 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:01.624781 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:01.630857 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 234/485] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Sep 22 02:33:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:33:01.640112 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5531b-4c09-405f-9257-4778d20cc195 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:01.691551 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 252/486] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 02:33:01.778437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5531b-4c09-405f-9257-4778d20cc195 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:01.815797 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724618f0-ea67-4fe3-9df2-081a98fba2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:01.816813 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724618f0-ea67-4fe3-9df2-081a98fba2c1 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:01.817293 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724618f0-ea67-4fe3-9df2-081a98fba2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:01.817761 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724618f0-ea67-4fe3-9df2-081a98fba2c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:01.832384 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724618f0-ea67-4fe3-9df2-081a98fba2c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:01.834211 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724618f0-ea67-4fe3-9df2-081a98fba2c1 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:01.850211 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-724618f0-ea67-4fe3-9df2-081a98fba2c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:01.850211 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:01.850211 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:01.850211 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:01.850211 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:01.850211 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:01.850211 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:01.850211 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:01.850211 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:01.850211 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:01.850211 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:01.850211 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:01.850211 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:01.850211 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:01.850211 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:01.850211 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:01.851801 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:01.851801 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:01.851801 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:01.851801 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:01.851801 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:01.851801 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:01.851801 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:01.851801 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:01.851801 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:01.851801 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:01.851801 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:01.851801 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:01.851801 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:01.851801 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:01.851801 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:01.851801 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:01.851801 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:01.852673 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:01.852673 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:01.852673 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:01.854799 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 235/487] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:01.860669 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc59b4-3f13-45e1-b0e8-bea3763100b0 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:02.004057 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:02.008093 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc59b4-3f13-45e1-b0e8-bea3763100b0 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:02.066484 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e86620-0fb2-48d4-8dd6-8faf8f72031b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:02.067228 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e86620-0fb2-48d4-8dd6-8faf8f72031b None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:02.067513 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e86620-0fb2-48d4-8dd6-8faf8f72031b None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:02.067792 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e86620-0fb2-48d4-8dd6-8faf8f72031b None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:02.068458 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e86620-0fb2-48d4-8dd6-8faf8f72031b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:02.068772 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e86620-0fb2-48d4-8dd6-8faf8f72031b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:02.070216 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e86620-0fb2-48d4-8dd6-8faf8f72031b None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:02.087666 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 253/488] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:02.095965 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724618f0-ea67-4fe3-9df2-081a98fba2c1 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:02.113351 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e86620-0fb2-48d4-8dd6-8faf8f72031b None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:02.213719 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:02.311580 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724618f0-ea67-4fe3-9df2-081a98fba2c1 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:02.333577 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e86620-0fb2-48d4-8dd6-8faf8f72031b None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:02.361893 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5173e482-abb8-4e62-9327-7c44826bbce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:02.362590 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173e482-abb8-4e62-9327-7c44826bbce6 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:02.363959 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173e482-abb8-4e62-9327-7c44826bbce6 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:02.364219 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173e482-abb8-4e62-9327-7c44826bbce6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:02.381150 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5173e482-abb8-4e62-9327-7c44826bbce6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:02.387376 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5173e482-abb8-4e62-9327-7c44826bbce6 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:02.414008 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-5173e482-abb8-4e62-9327-7c44826bbce6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 02:33:02.414008 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:02.414008 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:02.414008 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:02.414008 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:02.414008 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:02.414008 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:02.414008 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:02.414008 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:02.414008 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:02.414008 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:02.414008 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:02.414008 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:33:02.414008 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:33:02.414008 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:33:02.414008 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:33:02.415009 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:02.415009 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:02.415009 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:33:02.415009 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:02.415009 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:33:02.415009 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:02.415009 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:02.415009 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:02.415009 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:02.415009 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:02.415009 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:02.415009 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:02.415009 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:02.415009 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:02.415009 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:02.415009 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:02.415009 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:02.415009 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:02.415925 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:02.415925 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:02.415925 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:33:02.415925 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:33:02.415925 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:33:02.415925 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:33:02.415925 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:33:02.415925 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:33:02.415925 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 02:33:02.415925 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:02.418435 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 236/489] 10.222.0.20 () {56 vars in 1093 bytes} [Thu Sep 22 02:33:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:02.419668 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9bd3a-d74e-4abc-9932-b744c495abc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:02.420329 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9bd3a-d74e-4abc-9932-b744c495abc9 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:02.420605 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9bd3a-d74e-4abc-9932-b744c495abc9 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:02.420887 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9bd3a-d74e-4abc-9932-b744c495abc9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:02.423918 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5173e482-abb8-4e62-9327-7c44826bbce6 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:02.445754 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc9bd3a-d74e-4abc-9932-b744c495abc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:33:02.451183 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-adc9bd3a-d74e-4abc-9932-b744c495abc9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:33:02.451183 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:02.451183 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:02.451183 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:02.451183 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:02.451183 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:02.451183 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:02.451183 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:02.451183 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:02.451183 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:02.451183 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:02.451183 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:02.451183 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:33:02.451183 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:33:02.451183 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:33:02.451183 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:33:02.453327 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:33:02.453327 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:33:02.453327 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:33:02.453327 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:33:02.453327 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:02.453327 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:02.453327 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:02.453327 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:02.453327 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:02.453327 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:02.453327 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:02.453327 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:02.453327 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:02.453327 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:02.453327 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:02.453327 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:02.455264 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:02.455264 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:02.455264 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:02.455264 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:02.455264 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:33:02.455264 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:33:02.455264 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:33:02.455264 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:33:02.455264 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:33:02.455264 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:33:02.455264 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:33:02.455264 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:02.455264 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 254/490] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Sep 22 02:33:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:02.463225 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9bd3a-d74e-4abc-9932-b744c495abc9 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:02.506310 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:02.600035 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5173e482-abb8-4e62-9327-7c44826bbce6 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:02.603783 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9bd3a-d74e-4abc-9932-b744c495abc9 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:02.617114 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:02.644036 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb18db04-9a4e-4e41-9c45-d59c79497d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:02.644759 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18db04-9a4e-4e41-9c45-d59c79497d45 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:02.645119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18db04-9a4e-4e41-9c45-d59c79497d45 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:02.645438 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18db04-9a4e-4e41-9c45-d59c79497d45 None admin] PATH_INFO: `/v3/users` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:02.646150 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb18db04-9a4e-4e41-9c45-d59c79497d45 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:02.646461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb18db04-9a4e-4e41-9c45-d59c79497d45 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:02.646634 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edf0549-c913-41d5-8720-84ea8f46fff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:02.647376 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf0549-c913-41d5-8720-84ea8f46fff0 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:02.647663 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf0549-c913-41d5-8720-84ea8f46fff0 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:02.647940 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf0549-c913-41d5-8720-84ea8f46fff0 None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:02.648117 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb18db04-9a4e-4e41-9c45-d59c79497d45 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:02.649306 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edf0549-c913-41d5-8720-84ea8f46fff0 None admin] RBAC: Adding query filter params (name=service) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:02.649615 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edf0549-c913-41d5-8720-84ea8f46fff0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:02.651117 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edf0549-c913-41d5-8720-84ea8f46fff0 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:02.666372 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 255/491] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Sep 22 02:33:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:33:02.669333 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce4db8f-19b6-4421-b87d-891217a9143e None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:02.669831 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce4db8f-19b6-4421-b87d-891217a9143e None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:02.670316 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce4db8f-19b6-4421-b87d-891217a9143e None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:02.672942 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 256/492] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:33:02.676355 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c6e73-5970-452a-a002-77ef481907b7 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:02.678006 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c6e73-5970-452a-a002-77ef481907b7 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:02.678544 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c6e73-5970-452a-a002-77ef481907b7 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:02.680283 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 257/493] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:33:02.681266 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c6e73-5970-452a-a002-77ef481907b7 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:02.692079 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 237/494] 10.222.0.20 () {56 vars in 1133 bytes} [Thu Sep 22 02:33:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:02.693579 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84643b2-a66e-4e0d-bdcd-4866de2476ae None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:02.693873 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84643b2-a66e-4e0d-bdcd-4866de2476ae None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:02.694153 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84643b2-a66e-4e0d-bdcd-4866de2476ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:02.715690 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-a84643b2-a66e-4e0d-bdcd-4866de2476ae None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:02.716931 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-a84643b2-a66e-4e0d-bdcd-4866de2476ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:02.791959 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 238/495] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:33:02.793632 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f05c3-7ee6-4dc3-a58e-9ba4d2d1aa83 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:02.794139 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f05c3-7ee6-4dc3-a58e-9ba4d2d1aa83 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:02.794621 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f05c3-7ee6-4dc3-a58e-9ba4d2d1aa83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:02.814644 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-588f05c3-7ee6-4dc3-a58e-9ba4d2d1aa83 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:02.816240 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-588f05c3-7ee6-4dc3-a58e-9ba4d2d1aa83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:02.840437 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:02.864859 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c6e73-5970-452a-a002-77ef481907b7 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:02.906260 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7bf6a6-d824-422e-a032-e253f27b2fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:02.906566 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7bf6a6-d824-422e-a032-e253f27b2fb2 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:02.906881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7bf6a6-d824-422e-a032-e253f27b2fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:02.907169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7bf6a6-d824-422e-a032-e253f27b2fb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:02.913280 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 239/496] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:33:02.915185 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f05c3-7ee6-4dc3-a58e-9ba4d2d1aa83 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:02.921082 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7bf6a6-d824-422e-a032-e253f27b2fb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:02.922565 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7bf6a6-d824-422e-a032-e253f27b2fb2 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:02.940741 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-8b7bf6a6-d824-422e-a032-e253f27b2fb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:02.940741 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:02.940741 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:02.940741 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:02.940741 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:02.940741 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:02.940741 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:02.940741 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:02.940741 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:02.940741 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:02.940741 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:02.940741 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:02.940741 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:02.940741 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:02.940741 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:02.940741 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:02.941801 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:02.941801 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:02.941801 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:02.941801 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:02.941801 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:02.941801 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:02.941801 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:02.941801 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:02.941801 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:02.941801 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:02.941801 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:02.941801 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:02.941801 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:02.941801 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:02.941801 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:02.941801 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:02.941801 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:02.942646 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:02.942646 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:02.942646 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:02.942646 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 258/497] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:02.949554 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7bf6a6-d824-422e-a032-e253f27b2fb2 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:02.976974 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:03.073104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7bf6a6-d824-422e-a032-e253f27b2fb2 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:03.116272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b64d12-ceef-451b-93ca-61e1dd299281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:03.117480 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b64d12-ceef-451b-93ca-61e1dd299281 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:03.117972 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b64d12-ceef-451b-93ca-61e1dd299281 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:03.118478 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b64d12-ceef-451b-93ca-61e1dd299281 None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:03.119378 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b64d12-ceef-451b-93ca-61e1dd299281 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:03.119894 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b64d12-ceef-451b-93ca-61e1dd299281 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:03.121554 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b64d12-ceef-451b-93ca-61e1dd299281 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:03.123518 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:03.138513 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 259/498] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:33:03.145297 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b64d12-ceef-451b-93ca-61e1dd299281 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:03.160635 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f05c3-7ee6-4dc3-a58e-9ba4d2d1aa83 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:03.252188 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00204eca-273c-4641-bee5-c30076a24fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:03.252890 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00204eca-273c-4641-bee5-c30076a24fe0 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:03.253167 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00204eca-273c-4641-bee5-c30076a24fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:03.253646 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00204eca-273c-4641-bee5-c30076a24fe0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:03.266557 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b64d12-ceef-451b-93ca-61e1dd299281 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:03.309254 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0b14a-464e-4b68-82e1-21eae497f15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:03.310354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0b14a-464e-4b68-82e1-21eae497f15b None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:03.310907 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0b14a-464e-4b68-82e1-21eae497f15b None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:03.311654 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0b14a-464e-4b68-82e1-21eae497f15b None admin] PATH_INFO: `/v3/users/glance` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:03.321631 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb0b14a-464e-4b68-82e1-21eae497f15b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:03.323704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb0b14a-464e-4b68-82e1-21eae497f15b None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:03.343920 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-5eb0b14a-464e-4b68-82e1-21eae497f15b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 22 02:33:03.343920 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:03.343920 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:03.343920 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:03.343920 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:03.343920 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:03.343920 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:03.343920 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:03.343920 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:03.343920 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:03.343920 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:03.343920 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:03.343920 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:33:03.343920 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:33:03.343920 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:33:03.343920 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:33:03.345622 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:03.345622 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:03.345622 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:33:03.345622 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:03.345622 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:33:03.345622 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:03.345622 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:03.345622 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:03.345622 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:03.345622 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:03.345622 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:03.345622 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:03.345622 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:03.345622 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:03.345622 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:03.345622 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:03.345622 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:03.345622 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:03.347293 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:03.347293 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:03.347293 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:33:03.347293 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:33:03.347293 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:33:03.347293 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:33:03.347293 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:33:03.347293 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:33:03.347293 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 22 02:33:03.347293 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:03.352810 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 260/499] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:33:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:33:03.363335 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0b14a-464e-4b68-82e1-21eae497f15b None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:03.427733 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00204eca-273c-4641-bee5-c30076a24fe0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:03.431089 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00204eca-273c-4641-bee5-c30076a24fe0 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:03.483494 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:03.484292 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:03.566666 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0b14a-464e-4b68-82e1-21eae497f15b None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:03.612928 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 240/500] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:33:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 692 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 02:33:03.612928 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00204eca-273c-4641-bee5-c30076a24fe0 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:03.622571 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746fe8be-1493-44fb-8cdc-f45e105f6f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:03.623958 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746fe8be-1493-44fb-8cdc-f45e105f6f1e None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:03.624315 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746fe8be-1493-44fb-8cdc-f45e105f6f1e None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:03.624597 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746fe8be-1493-44fb-8cdc-f45e105f6f1e None admin] PATH_INFO: `/v3/users` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:03.625465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746fe8be-1493-44fb-8cdc-f45e105f6f1e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:03.625788 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746fe8be-1493-44fb-8cdc-f45e105f6f1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:03.627956 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746fe8be-1493-44fb-8cdc-f45e105f6f1e None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:03.677160 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 261/501] 10.222.0.20 () {56 vars in 1137 bytes} [Thu Sep 22 02:33:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:33:03.678120 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746fe8be-1493-44fb-8cdc-f45e105f6f1e None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:03.778077 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00204eca-273c-4641-bee5-c30076a24fe0 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:03.815469 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:03.816239 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746fe8be-1493-44fb-8cdc-f45e105f6f1e None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:03.825187 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855252f5-2a43-444a-8aaa-70b1030eac45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:03.825523 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855252f5-2a43-444a-8aaa-70b1030eac45 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:03.827154 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855252f5-2a43-444a-8aaa-70b1030eac45 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:03.827154 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855252f5-2a43-444a-8aaa-70b1030eac45 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:03.831411 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:03.840886 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855252f5-2a43-444a-8aaa-70b1030eac45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:33:03.845911 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-855252f5-2a43-444a-8aaa-70b1030eac45 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:33:03.845911 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:03.845911 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:03.845911 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:03.845911 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:03.845911 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:03.845911 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:03.845911 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:03.845911 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:03.845911 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:03.845911 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:03.845911 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:03.845911 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:33:03.845911 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:33:03.845911 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:33:03.845911 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:33:03.847815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:33:03.847815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:33:03.847815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:33:03.847815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:33:03.847815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:03.847815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:03.847815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:03.847815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:03.847815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:03.847815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:03.847815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:03.847815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:03.847815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:03.847815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:03.847815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:03.847815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:03.849674 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:03.849674 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:03.849674 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:03.849674 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:03.849674 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:33:03.849674 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:33:03.849674 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:33:03.849674 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:33:03.849674 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:33:03.849674 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:33:03.849674 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:33:03.849674 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:03.849674 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 241/502] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Sep 22 02:33:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:03.849674 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855252f5-2a43-444a-8aaa-70b1030eac45 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:03.864459 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4053a24a-7149-43ec-8496-b49cf4e61ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:03.865505 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053a24a-7149-43ec-8496-b49cf4e61ecd None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:03.865505 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053a24a-7149-43ec-8496-b49cf4e61ecd None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:03.865810 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053a24a-7149-43ec-8496-b49cf4e61ecd None admin] PATH_INFO: `/v3/roles/service` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:03.880358 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4053a24a-7149-43ec-8496-b49cf4e61ecd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:33:03.886202 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-4053a24a-7149-43ec-8496-b49cf4e61ecd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:33:03.886202 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:03.886202 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:03.886202 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:03.886202 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:03.886202 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:03.886202 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:03.886202 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:03.886202 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:03.886202 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:03.886202 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:03.886202 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:03.886202 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:33:03.886202 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:33:03.886202 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:33:03.886202 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:33:03.887255 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:33:03.887255 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:33:03.887255 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:33:03.887255 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:33:03.887255 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:03.887255 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:03.887255 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:03.887255 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:03.887255 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:03.887255 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:03.887255 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:03.887255 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:03.887255 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:03.887255 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:03.887255 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:03.887255 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:03.888066 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:03.888066 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:03.888066 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:03.888066 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:03.888066 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:33:03.888066 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:33:03.888066 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:33:03.888066 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:33:03.888066 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:33:03.888066 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:33:03.888066 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:33:03.888066 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:03.888066 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 262/503] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Sep 22 02:33:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:03.889496 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4053a24a-7149-43ec-8496-b49cf4e61ecd None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:03.977682 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855252f5-2a43-444a-8aaa-70b1030eac45 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:04.005525 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4053a24a-7149-43ec-8496-b49cf4e61ecd None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:04.015035 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19328d6a-1313-4911-b4a7-17a086155359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:04.015688 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19328d6a-1313-4911-b4a7-17a086155359 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:04.015972 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19328d6a-1313-4911-b4a7-17a086155359 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:04.016250 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19328d6a-1313-4911-b4a7-17a086155359 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:04.027879 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19328d6a-1313-4911-b4a7-17a086155359 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:04.029356 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19328d6a-1313-4911-b4a7-17a086155359 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:04.044958 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-19328d6a-1313-4911-b4a7-17a086155359 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:04.044958 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:04.044958 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:04.044958 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:04.044958 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:04.044958 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:04.044958 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:04.044958 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:04.044958 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:04.044958 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:04.044958 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:04.044958 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:04.044958 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:04.044958 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:04.044958 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:04.044958 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:04.045949 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:04.045949 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:04.045949 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:04.045949 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:04.045949 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:04.045949 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:04.045949 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:04.045949 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:04.045949 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:04.045949 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:04.045949 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:04.045949 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:04.045949 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:04.045949 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:04.045949 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:04.045949 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:04.045949 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:04.046880 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:04.046880 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:04.046880 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:04.046880 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 242/504] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:04.047488 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f1167-e9e2-4ac9-b8e0-75d5955aa489 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:04.047644 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f1167-e9e2-4ac9-b8e0-75d5955aa489 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:04.047877 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f1167-e9e2-4ac9-b8e0-75d5955aa489 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:04.049276 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 243/505] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:33:04.051572 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f1167-e9e2-4ac9-b8e0-75d5955aa489 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:04.064004 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88feaa66-1e7c-490c-92cd-e19798ae7a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:04.064625 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88feaa66-1e7c-490c-92cd-e19798ae7a5c None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:04.064906 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88feaa66-1e7c-490c-92cd-e19798ae7a5c None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:04.065177 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88feaa66-1e7c-490c-92cd-e19798ae7a5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:04.106519 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:04.130098 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:04.169884 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f1167-e9e2-4ac9-b8e0-75d5955aa489 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:04.170926 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88feaa66-1e7c-490c-92cd-e19798ae7a5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:04.173000 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88feaa66-1e7c-490c-92cd-e19798ae7a5c None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:04.209173 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb4375-f95b-4263-b540-c14133b1b004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:04.210198 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb4375-f95b-4263-b540-c14133b1b004 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:04.210452 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb4375-f95b-4263-b540-c14133b1b004 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:04.210782 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb4375-f95b-4263-b540-c14133b1b004 None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:04.211523 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eb4375-f95b-4263-b540-c14133b1b004 None admin] RBAC: Adding query filter params (name=service) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:04.211841 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eb4375-f95b-4263-b540-c14133b1b004 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:04.213278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eb4375-f95b-4263-b540-c14133b1b004 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:04.222303 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 244/506] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Sep 22 02:33:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:04.223208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb4375-f95b-4263-b540-c14133b1b004 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:04.293389 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 263/507] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:33:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 02:33:04.294649 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b863bb3-4e35-434a-bd28-0d05e68c54f2 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:04.294971 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b863bb3-4e35-434a-bd28-0d05e68c54f2 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:04.295251 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b863bb3-4e35-434a-bd28-0d05e68c54f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:04.315334 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-5b863bb3-4e35-434a-bd28-0d05e68c54f2 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:04.316574 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-5b863bb3-4e35-434a-bd28-0d05e68c54f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:04.368140 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb4375-f95b-4263-b540-c14133b1b004 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:04.371525 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:04.399159 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 264/508] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:33:04.400111 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b863bb3-4e35-434a-bd28-0d05e68c54f2 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:04.402406 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:04.427823 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1180a8-0e02-4cd8-9f9f-4d24bf3c3446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:04.428385 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1180a8-0e02-4cd8-9f9f-4d24bf3c3446 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:04.428768 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1180a8-0e02-4cd8-9f9f-4d24bf3c3446 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:04.429305 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1180a8-0e02-4cd8-9f9f-4d24bf3c3446 None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:04.430556 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1180a8-0e02-4cd8-9f9f-4d24bf3c3446 None admin] RBAC: Adding query filter params (name=service) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:04.431154 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1180a8-0e02-4cd8-9f9f-4d24bf3c3446 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:04.433812 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1180a8-0e02-4cd8-9f9f-4d24bf3c3446 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:04.449128 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 245/509] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Sep 22 02:33:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:04.450543 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1180a8-0e02-4cd8-9f9f-4d24bf3c3446 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:04.550815 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b863bb3-4e35-434a-bd28-0d05e68c54f2 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:04.584681 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1180a8-0e02-4cd8-9f9f-4d24bf3c3446 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:04.593486 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950aaab0-c22c-460b-9260-3d1fd74bb732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:04.594102 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950aaab0-c22c-460b-9260-3d1fd74bb732 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:04.594390 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950aaab0-c22c-460b-9260-3d1fd74bb732 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:04.594657 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950aaab0-c22c-460b-9260-3d1fd74bb732 None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:04.595344 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950aaab0-c22c-460b-9260-3d1fd74bb732 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:04.595645 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950aaab0-c22c-460b-9260-3d1fd74bb732 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:04.597042 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950aaab0-c22c-460b-9260-3d1fd74bb732 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:04.609862 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 265/510] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:04.610733 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950aaab0-c22c-460b-9260-3d1fd74bb732 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:04.647762 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08ed73-930f-43cb-b28a-1f3ff415d1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:04.650003 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08ed73-930f-43cb-b28a-1f3ff415d1f7 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:04.650408 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08ed73-930f-43cb-b28a-1f3ff415d1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:04.650898 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08ed73-930f-43cb-b28a-1f3ff415d1f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:04.670033 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08ed73-930f-43cb-b28a-1f3ff415d1f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:04.672067 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08ed73-930f-43cb-b28a-1f3ff415d1f7 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:04.689110 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:04.690660 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-ff08ed73-930f-43cb-b28a-1f3ff415d1f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:04.690660 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:04.690660 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:04.690660 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:04.690660 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:04.690660 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:04.690660 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:04.690660 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:04.690660 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:04.690660 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:04.690660 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:04.690660 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:04.690660 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:04.690660 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:04.690660 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:04.690660 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:04.692512 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:04.692512 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:04.692512 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:04.692512 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:04.692512 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:04.692512 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:04.692512 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:04.692512 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:04.692512 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:04.692512 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:04.692512 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:04.692512 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:04.692512 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:04.692512 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:04.692512 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:04.692512 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:04.692512 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:04.694459 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:04.694459 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:04.694459 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:04.694459 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 246/511] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:04.694459 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08ed73-930f-43cb-b28a-1f3ff415d1f7 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:04.726965 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:04.737275 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950aaab0-c22c-460b-9260-3d1fd74bb732 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:04.776652 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a297c87a-6f61-44a5-a47f-5524bc7014fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:04.777291 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297c87a-6f61-44a5-a47f-5524bc7014fa None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:04.777563 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297c87a-6f61-44a5-a47f-5524bc7014fa None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:04.777833 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297c87a-6f61-44a5-a47f-5524bc7014fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:04.792199 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a297c87a-6f61-44a5-a47f-5524bc7014fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:04.793671 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a297c87a-6f61-44a5-a47f-5524bc7014fa None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:04.816201 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-a297c87a-6f61-44a5-a47f-5524bc7014fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:04.816201 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:04.816201 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:04.816201 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:04.816201 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:04.816201 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:04.816201 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:04.816201 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:04.816201 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:04.816201 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:04.816201 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:04.816201 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:04.816201 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:04.816201 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:04.816201 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:04.816201 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:04.818347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:04.818347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:04.818347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:04.818347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:04.818347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:04.818347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:04.818347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:04.818347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:04.818347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:04.818347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:04.818347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:04.818347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:04.818347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:04.818347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:04.818347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:04.818347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:04.818347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:04.820353 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:04.820353 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:04.820353 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:04.820353 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 266/512] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:04.820353 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a297c87a-6f61-44a5-a47f-5524bc7014fa None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:04.824026 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08ed73-930f-43cb-b28a-1f3ff415d1f7 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:04.864329 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcdde6c-358f-45c3-be8c-c2bcfd29431d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:04.865376 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdde6c-358f-45c3-be8c-c2bcfd29431d None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:04.865376 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdde6c-358f-45c3-be8c-c2bcfd29431d None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:04.865580 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdde6c-358f-45c3-be8c-c2bcfd29431d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:04.877045 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcdde6c-358f-45c3-be8c-c2bcfd29431d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:04.878630 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcdde6c-358f-45c3-be8c-c2bcfd29431d None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:04.894099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-afcdde6c-358f-45c3-be8c-c2bcfd29431d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:04.894099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:04.894099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:04.894099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:04.894099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:04.894099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:04.894099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:04.894099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:04.894099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:04.894099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:04.894099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:04.894099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:04.894099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:04.894099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:04.894099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:04.894099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:04.895903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:04.895903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:04.895903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:04.895903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:04.895903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:04.895903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:04.895903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:04.895903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:04.895903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:04.895903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:04.895903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:04.895903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:04.895903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:04.895903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:04.895903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:04.895903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:04.895903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:04.896792 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:04.896792 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:04.896792 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:04.896792 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 247/513] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:04.896792 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcdde6c-358f-45c3-be8c-c2bcfd29431d None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:04.940894 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a297c87a-6f61-44a5-a47f-5524bc7014fa None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:04.955603 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:04.975996 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:04.981494 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033059de-9ee0-4d43-8488-1157bb3ccdb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:04.982346 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033059de-9ee0-4d43-8488-1157bb3ccdb0 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:04.982629 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033059de-9ee0-4d43-8488-1157bb3ccdb0 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:04.983045 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033059de-9ee0-4d43-8488-1157bb3ccdb0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:04.990760 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033059de-9ee0-4d43-8488-1157bb3ccdb0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:04.992316 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033059de-9ee0-4d43-8488-1157bb3ccdb0 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:05.009276 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-033059de-9ee0-4d43-8488-1157bb3ccdb0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:05.009276 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:05.009276 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:05.009276 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:05.009276 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:05.009276 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:05.009276 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:05.009276 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:05.009276 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:05.009276 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:05.009276 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:05.009276 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:05.009276 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:33:05.009276 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:33:05.009276 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:33:05.010227 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:33:05.010227 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:05.010227 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:05.010227 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:05.010227 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:05.010227 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:05.010227 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:05.010227 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:05.010227 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:05.010227 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:05.010227 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:05.010227 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:05.010227 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:05.010227 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:05.010227 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:05.010227 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:05.010227 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:05.011145 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:33:05.011145 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:33:05.011145 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:33:05.011145 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:33:05.011145 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:33:05.011145 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:33:05.011145 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:05.011145 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:05.011576 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcdde6c-358f-45c3-be8c-c2bcfd29431d None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:05.012087 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 267/514] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:33:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:05.012572 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033059de-9ee0-4d43-8488-1157bb3ccdb0 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:05.058301 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852b876-1112-492a-8fc8-adb38a24d2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:05.058975 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852b876-1112-492a-8fc8-adb38a24d2e0 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:05.059266 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852b876-1112-492a-8fc8-adb38a24d2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:05.059545 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852b876-1112-492a-8fc8-adb38a24d2e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:05.071287 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852b876-1112-492a-8fc8-adb38a24d2e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:05.072787 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852b876-1112-492a-8fc8-adb38a24d2e0 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:05.088636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-b852b876-1112-492a-8fc8-adb38a24d2e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:05.088636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:05.088636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:05.088636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:05.088636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:05.088636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:05.088636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:05.088636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:05.088636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:05.088636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:05.088636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:05.088636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:05.088636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:05.088636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:05.088636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:05.088636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:05.089716 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:05.089716 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:05.089716 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:05.089716 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:05.089716 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:05.089716 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:05.089716 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:05.089716 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:05.089716 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:05.089716 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:05.089716 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:05.089716 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:05.089716 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:05.089716 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:05.089716 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:05.089716 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:05.089716 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:05.090722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:05.090722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:05.090722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:05.090722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 248/515] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:05.090722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852b876-1112-492a-8fc8-adb38a24d2e0 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:05.136865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033059de-9ee0-4d43-8488-1157bb3ccdb0 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:05.175859 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c1102-2004-4893-b033-e30db07b880b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:05.176556 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c1102-2004-4893-b033-e30db07b880b None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:05.176866 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c1102-2004-4893-b033-e30db07b880b None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:05.177275 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c1102-2004-4893-b033-e30db07b880b None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:05.177706 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33c1102-2004-4893-b033-e30db07b880b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:05.177996 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33c1102-2004-4893-b033-e30db07b880b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:05.179535 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33c1102-2004-4893-b033-e30db07b880b None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:05.192724 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 268/516] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:05.193590 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c1102-2004-4893-b033-e30db07b880b None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:05.203492 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852b876-1112-492a-8fc8-adb38a24d2e0 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:05.228411 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:05.230197 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:05.241363 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc25407-eb42-4be9-b874-fda7d5b18405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:05.242485 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc25407-eb42-4be9-b874-fda7d5b18405 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:05.242769 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc25407-eb42-4be9-b874-fda7d5b18405 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:05.243076 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc25407-eb42-4be9-b874-fda7d5b18405 None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:05.243730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc25407-eb42-4be9-b874-fda7d5b18405 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:05.244043 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc25407-eb42-4be9-b874-fda7d5b18405 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:05.245440 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc25407-eb42-4be9-b874-fda7d5b18405 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:05.264104 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 249/517] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:05.264971 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc25407-eb42-4be9-b874-fda7d5b18405 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:05.302270 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c1102-2004-4893-b033-e30db07b880b None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:05.339357 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92934f5e-a24b-42f1-a3cd-0b72858c7ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:05.339882 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92934f5e-a24b-42f1-a3cd-0b72858c7ac6 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:05.340167 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92934f5e-a24b-42f1-a3cd-0b72858c7ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:05.340443 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92934f5e-a24b-42f1-a3cd-0b72858c7ac6 None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:05.341099 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92934f5e-a24b-42f1-a3cd-0b72858c7ac6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:05.341406 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92934f5e-a24b-42f1-a3cd-0b72858c7ac6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:05.342798 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92934f5e-a24b-42f1-a3cd-0b72858c7ac6 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:05.354881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 269/518] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:05.355746 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92934f5e-a24b-42f1-a3cd-0b72858c7ac6 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:05.384297 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc25407-eb42-4be9-b874-fda7d5b18405 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:05.421334 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679710ff-740d-4f87-a5bf-763ba9f9a18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:05.421948 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679710ff-740d-4f87-a5bf-763ba9f9a18d None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:05.422231 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679710ff-740d-4f87-a5bf-763ba9f9a18d None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:05.422505 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679710ff-740d-4f87-a5bf-763ba9f9a18d None admin] PATH_INFO: `/v3/projects` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:05.423248 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679710ff-740d-4f87-a5bf-763ba9f9a18d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:05.423850 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679710ff-740d-4f87-a5bf-763ba9f9a18d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:05.425025 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679710ff-740d-4f87-a5bf-763ba9f9a18d None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:05.443331 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 250/519] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Sep 22 02:33:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:05.444675 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679710ff-740d-4f87-a5bf-763ba9f9a18d None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:05.477236 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:05.486939 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92934f5e-a24b-42f1-a3cd-0b72858c7ac6 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:05.494101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:05.528886 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77462401-eb38-4ac1-9de0-fd0cab5d0d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:05.529791 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77462401-eb38-4ac1-9de0-fd0cab5d0d33 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:05.530079 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77462401-eb38-4ac1-9de0-fd0cab5d0d33 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:05.530369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77462401-eb38-4ac1-9de0-fd0cab5d0d33 None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:05.531077 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77462401-eb38-4ac1-9de0-fd0cab5d0d33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:05.531388 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77462401-eb38-4ac1-9de0-fd0cab5d0d33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:05.532813 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77462401-eb38-4ac1-9de0-fd0cab5d0d33 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:05.546792 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 270/520] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:05.547690 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77462401-eb38-4ac1-9de0-fd0cab5d0d33 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:05.559770 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679710ff-740d-4f87-a5bf-763ba9f9a18d None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:05.599402 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65d304-2826-4973-a92d-fc231da1088e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:05.599947 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65d304-2826-4973-a92d-fc231da1088e None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:05.600220 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65d304-2826-4973-a92d-fc231da1088e None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:05.600492 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65d304-2826-4973-a92d-fc231da1088e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:05.611881 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca65d304-2826-4973-a92d-fc231da1088e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:05.613369 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca65d304-2826-4973-a92d-fc231da1088e None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:05.626176 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-ca65d304-2826-4973-a92d-fc231da1088e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 22 02:33:05.626176 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:05.626176 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:05.626176 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:05.626176 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:05.626176 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:05.626176 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:05.626176 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:05.626176 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:05.626176 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:05.626176 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:05.626176 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:05.626176 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:33:05.626176 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:33:05.626176 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:33:05.626176 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:33:05.627456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:05.627456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:05.627456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:33:05.627456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:05.627456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:33:05.627456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:05.627456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:05.627456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:05.627456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:05.627456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:05.627456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:05.627456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:05.627456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:05.627456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:05.627456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:05.627456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:05.627456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:05.627456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:05.628601 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:05.628601 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:05.628601 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:33:05.628601 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:33:05.628601 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:33:05.628601 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:33:05.628601 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:33:05.628601 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:33:05.628601 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 22 02:33:05.628601 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:05.628601 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 251/521] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Sep 22 02:33:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:05.628601 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65d304-2826-4973-a92d-fc231da1088e None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:05.675207 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77462401-eb38-4ac1-9de0-fd0cab5d0d33 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:05.715355 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a44a2e5-9151-474a-8d54-406b614974bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:05.716003 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44a2e5-9151-474a-8d54-406b614974bd None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:05.716290 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44a2e5-9151-474a-8d54-406b614974bd None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:05.716568 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44a2e5-9151-474a-8d54-406b614974bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:05.723829 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a44a2e5-9151-474a-8d54-406b614974bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:05.725322 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a44a2e5-9151-474a-8d54-406b614974bd None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:05.738893 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-7a44a2e5-9151-474a-8d54-406b614974bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:05.738893 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:05.738893 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:05.738893 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:05.738893 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:05.738893 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:05.738893 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:05.738893 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:05.738893 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:05.738893 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:05.738893 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:05.738893 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:05.738893 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:33:05.738893 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:33:05.738893 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:33:05.739852 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:33:05.739852 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:05.739852 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:05.739852 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:05.739852 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:05.739852 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:05.739852 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:05.739852 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:05.739852 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:05.739852 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:05.739852 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:05.739852 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:05.739852 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:05.739852 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:05.739852 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:05.739852 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:05.739852 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:05.740700 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:33:05.740700 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:33:05.740700 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:33:05.740700 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:33:05.740700 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:33:05.740700 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:33:05.740700 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:05.740700 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:05.740700 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 271/522] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:33:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:05.741165 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a44a2e5-9151-474a-8d54-406b614974bd None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:05.748967 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65d304-2826-4973-a92d-fc231da1088e None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:05.750663 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:05.764278 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:05.793837 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b9904-fa20-422a-9a77-bac7a5410378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:05.794473 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b9904-fa20-422a-9a77-bac7a5410378 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:05.794754 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b9904-fa20-422a-9a77-bac7a5410378 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:05.795059 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b9904-fa20-422a-9a77-bac7a5410378 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:05.803584 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995b9904-fa20-422a-9a77-bac7a5410378 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:05.805638 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995b9904-fa20-422a-9a77-bac7a5410378 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:05.817989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-995b9904-fa20-422a-9a77-bac7a5410378 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 02:33:05.817989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:05.817989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:05.817989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:05.817989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:05.817989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:05.817989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:05.817989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:05.817989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:05.817989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:05.817989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:05.817989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:05.817989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:33:05.817989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:33:05.817989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:33:05.817989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:33:05.819298 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:05.819298 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:05.819298 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:33:05.819298 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:05.819298 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:33:05.819298 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:05.819298 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:05.819298 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:05.819298 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:05.819298 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:05.819298 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:05.819298 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:05.819298 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:05.819298 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:05.819298 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:05.819298 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:05.819298 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:05.819298 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:05.820430 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:05.820430 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:05.820430 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:33:05.820430 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:33:05.820430 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:33:05.820430 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:33:05.820430 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:33:05.820430 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:33:05.820430 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 02:33:05.820430 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:05.820430 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 252/523] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:33:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:05.820430 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b9904-fa20-422a-9a77-bac7a5410378 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:05.860564 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a44a2e5-9151-474a-8d54-406b614974bd None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:05.899064 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798896c-5534-4d09-b267-16945743a0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:05.899702 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798896c-5534-4d09-b267-16945743a0c3 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:05.899985 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798896c-5534-4d09-b267-16945743a0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:05.900259 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798896c-5534-4d09-b267-16945743a0c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:05.901000 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2798896c-5534-4d09-b267-16945743a0c3 None admin] RBAC: Adding query filter params (role.id=fb2fbee669e14ffbbe0aa70642eab1ce, scope.project.id=2a792f8dc45145d687ddd58c94d44b5e, user.id=a64ed6c8e15d479887ab1dc59997d715) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:05.901315 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2798896c-5534-4d09-b267-16945743a0c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:05.902772 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2798896c-5534-4d09-b267-16945743a0c3 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:05.908634 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 272/524] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:33:05 2022] GET /identity/v3/role_assignments?user.id=a64ed6c8e15d479887ab1dc59997d715&scope.project.id=2a792f8dc45145d687ddd58c94d44b5e&role.id=fb2fbee669e14ffbbe0aa70642eab1ce => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:05.909543 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798896c-5534-4d09-b267-16945743a0c3 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:05.945516 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b9904-fa20-422a-9a77-bac7a5410378 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:05.994453 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93480e5b-2ac1-4ad8-af6d-5a9313059dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:05.995116 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93480e5b-2ac1-4ad8-af6d-5a9313059dda None admin] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:05.996058 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93480e5b-2ac1-4ad8-af6d-5a9313059dda None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:05.996356 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93480e5b-2ac1-4ad8-af6d-5a9313059dda None admin] PATH_INFO: `/v3/users` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:05.997052 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93480e5b-2ac1-4ad8-af6d-5a9313059dda None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:05.998552 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93480e5b-2ac1-4ad8-af6d-5a9313059dda None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:06.041118 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 253/525] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Sep 22 02:33:05 2022] POST /identity/v3/users => generated 247 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:33:06.041118 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93480e5b-2ac1-4ad8-af6d-5a9313059dda None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:06.050098 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:06.078656 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:06.083981 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798896c-5534-4d09-b267-16945743a0c3 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:06.133509 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd5d34d-b9ae-46e3-8ca1-ddca6f579afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:06.134226 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5d34d-b9ae-46e3-8ca1-ddca6f579afa None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:06.134524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5d34d-b9ae-46e3-8ca1-ddca6f579afa None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:06.134934 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5d34d-b9ae-46e3-8ca1-ddca6f579afa None admin] PATH_INFO: `/v3/users` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:06.135773 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd5d34d-b9ae-46e3-8ca1-ddca6f579afa None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:06.136175 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd5d34d-b9ae-46e3-8ca1-ddca6f579afa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:06.137746 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd5d34d-b9ae-46e3-8ca1-ddca6f579afa None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:06.168522 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 273/526] 10.222.0.20 () {56 vars in 1139 bytes} [Thu Sep 22 02:33:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:06.169937 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd5d34d-b9ae-46e3-8ca1-ddca6f579afa None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:06.205702 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93480e5b-2ac1-4ad8-af6d-5a9313059dda None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:06.280817 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757bf984-6bea-4749-aac1-12da0e3eef06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:06.282558 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bf984-6bea-4749-aac1-12da0e3eef06 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:06.283177 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bf984-6bea-4749-aac1-12da0e3eef06 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:06.283670 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bf984-6bea-4749-aac1-12da0e3eef06 None admin] PATH_INFO: `/v3/projects` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:06.285343 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757bf984-6bea-4749-aac1-12da0e3eef06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:06.286325 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757bf984-6bea-4749-aac1-12da0e3eef06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:06.289945 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757bf984-6bea-4749-aac1-12da0e3eef06 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:06.309987 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 254/527] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Sep 22 02:33:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:06.311250 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757bf984-6bea-4749-aac1-12da0e3eef06 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:06.332543 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd5d34d-b9ae-46e3-8ca1-ddca6f579afa None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:06.378445 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6539a3-1113-4778-a0a9-3013d496865d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:06.379107 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6539a3-1113-4778-a0a9-3013d496865d None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:06.379387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6539a3-1113-4778-a0a9-3013d496865d None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:06.379663 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6539a3-1113-4778-a0a9-3013d496865d None admin] PATH_INFO: `/v3/users` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:06.380406 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6539a3-1113-4778-a0a9-3013d496865d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:06.380720 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6539a3-1113-4778-a0a9-3013d496865d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:06.382167 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6539a3-1113-4778-a0a9-3013d496865d None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:06.385831 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:06.409393 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 274/528] 10.222.0.20 () {56 vars in 1137 bytes} [Thu Sep 22 02:33:06 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:06.411322 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6539a3-1113-4778-a0a9-3013d496865d None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:06.435709 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:06.466081 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757bf984-6bea-4749-aac1-12da0e3eef06 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:06.503661 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc32408-fd1f-4eb5-8fcf-997769204356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:06.504603 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc32408-fd1f-4eb5-8fcf-997769204356 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:06.505026 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc32408-fd1f-4eb5-8fcf-997769204356 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:06.505537 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc32408-fd1f-4eb5-8fcf-997769204356 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:06.550074 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6539a3-1113-4778-a0a9-3013d496865d None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:06.581514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc32408-fd1f-4eb5-8fcf-997769204356 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:06.583296 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc32408-fd1f-4eb5-8fcf-997769204356 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:06.595831 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce584d15-3c60-40a7-b04b-a7ef8e659aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:06.596591 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce584d15-3c60-40a7-b04b-a7ef8e659aa5 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:06.597090 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce584d15-3c60-40a7-b04b-a7ef8e659aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:06.597380 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce584d15-3c60-40a7-b04b-a7ef8e659aa5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:06.598203 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce584d15-3c60-40a7-b04b-a7ef8e659aa5 None admin] RBAC: Adding query filter params (role.id=fb2fbee669e14ffbbe0aa70642eab1ce, scope.project.id=2a792f8dc45145d687ddd58c94d44b5e, user.id=6160520562b74be3b25c49a30e990d8b) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:06.598594 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce584d15-3c60-40a7-b04b-a7ef8e659aa5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:06.600291 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce584d15-3c60-40a7-b04b-a7ef8e659aa5 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:06.615293 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 275/529] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:33:06 2022] GET /identity/v3/role_assignments?user.id=6160520562b74be3b25c49a30e990d8b&scope.project.id=2a792f8dc45145d687ddd58c94d44b5e&role.id=fb2fbee669e14ffbbe0aa70642eab1ce => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:06.616981 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce584d15-3c60-40a7-b04b-a7ef8e659aa5 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:06.679516 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:06.702421 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 255/530] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:33:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 02:33:06.709314 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc32408-fd1f-4eb5-8fcf-997769204356 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:06.800810 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:06.892628 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce584d15-3c60-40a7-b04b-a7ef8e659aa5 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:06.912883 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc32408-fd1f-4eb5-8fcf-997769204356 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:06.939996 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045edc39-209b-4cab-8e53-a28214dd77d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:06.940537 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045edc39-209b-4cab-8e53-a28214dd77d3 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:06.940817 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045edc39-209b-4cab-8e53-a28214dd77d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:06.941734 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045edc39-209b-4cab-8e53-a28214dd77d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:07.020101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f64223-f302-4b40-81bb-1cd30c24b987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:07.022246 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f64223-f302-4b40-81bb-1cd30c24b987 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:07.022246 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f64223-f302-4b40-81bb-1cd30c24b987 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:07.022246 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f64223-f302-4b40-81bb-1cd30c24b987 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:07.032125 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045edc39-209b-4cab-8e53-a28214dd77d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:07.033701 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045edc39-209b-4cab-8e53-a28214dd77d3 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:07.047804 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f64223-f302-4b40-81bb-1cd30c24b987 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:07.049370 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f64223-f302-4b40-81bb-1cd30c24b987 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:07.080739 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-f4f64223-f302-4b40-81bb-1cd30c24b987 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:07.080739 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:07.080739 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:07.080739 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:07.080739 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:07.080739 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:07.080739 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:07.080739 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:07.080739 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:07.080739 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:07.080739 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:07.080739 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:07.080739 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:07.080739 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:07.080739 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:07.080739 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:07.081722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:07.081722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:07.081722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:07.081722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:07.081722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:07.081722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:07.081722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:07.081722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:07.081722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:07.081722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:07.081722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:07.081722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:07.081722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:07.081722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:07.081722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:07.081722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:07.081722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:07.082937 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:07.082937 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:07.082937 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:07.082937 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 256/531] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:07.091959 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f64223-f302-4b40-81bb-1cd30c24b987 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:07.108050 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:07.165014 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 276/532] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:33:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 02:33:07.174949 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045edc39-209b-4cab-8e53-a28214dd77d3 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:07.203594 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:07.393106 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f64223-f302-4b40-81bb-1cd30c24b987 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:07.421677 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045edc39-209b-4cab-8e53-a28214dd77d3 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:07.470642 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f531d5ae-65f8-4f8a-92c1-081f85c8cb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:07.471212 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f531d5ae-65f8-4f8a-92c1-081f85c8cb5a None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:07.471492 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f531d5ae-65f8-4f8a-92c1-081f85c8cb5a None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:07.471769 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f531d5ae-65f8-4f8a-92c1-081f85c8cb5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:07.473826 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf037395-fb1a-4932-8592-926bc1778290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:07.474525 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf037395-fb1a-4932-8592-926bc1778290 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:07.474805 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf037395-fb1a-4932-8592-926bc1778290 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:07.475166 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf037395-fb1a-4932-8592-926bc1778290 None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:07.476036 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf037395-fb1a-4932-8592-926bc1778290 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:07.476843 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf037395-fb1a-4932-8592-926bc1778290 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:07.480018 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf037395-fb1a-4932-8592-926bc1778290 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:07.499264 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f531d5ae-65f8-4f8a-92c1-081f85c8cb5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:07.501816 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f531d5ae-65f8-4f8a-92c1-081f85c8cb5a None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:07.504477 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 257/533] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:07.513504 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf037395-fb1a-4932-8592-926bc1778290 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:07.530930 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-f531d5ae-65f8-4f8a-92c1-081f85c8cb5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:07.530930 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:07.530930 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:07.530930 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:07.530930 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:07.530930 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:07.530930 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:07.530930 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:07.530930 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:07.530930 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:07.530930 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:07.530930 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:07.530930 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:07.530930 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:07.530930 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:07.530930 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:07.533323 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:07.533323 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:07.533323 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:07.533323 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:07.533323 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:07.533323 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:07.533323 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:07.533323 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:07.533323 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:07.533323 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:07.533323 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:07.533323 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:07.533323 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:07.533323 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:07.533323 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:07.533323 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:07.533323 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:07.536555 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:07.536555 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:07.536555 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:07.536555 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 277/534] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:07.546377 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f531d5ae-65f8-4f8a-92c1-081f85c8cb5a None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:07.572564 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:07.708117 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f531d5ae-65f8-4f8a-92c1-081f85c8cb5a None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:07.747009 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:07.753354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a8aec2-65ab-4f6f-a008-88c04cab9f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:07.754548 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8aec2-65ab-4f6f-a008-88c04cab9f30 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:07.754870 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8aec2-65ab-4f6f-a008-88c04cab9f30 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:07.755204 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8aec2-65ab-4f6f-a008-88c04cab9f30 None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:07.755884 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a8aec2-65ab-4f6f-a008-88c04cab9f30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:07.756193 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a8aec2-65ab-4f6f-a008-88c04cab9f30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:07.757665 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a8aec2-65ab-4f6f-a008-88c04cab9f30 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:07.771871 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf037395-fb1a-4932-8592-926bc1778290 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:07.785813 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 278/535] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:33:07.800515 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a8aec2-65ab-4f6f-a008-88c04cab9f30 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:07.875430 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e66ba-7358-4ba6-b72e-21eb67c6fbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:07.877147 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e66ba-7358-4ba6-b72e-21eb67c6fbe3 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:07.877988 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e66ba-7358-4ba6-b72e-21eb67c6fbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:07.878753 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e66ba-7358-4ba6-b72e-21eb67c6fbe3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:07.903866 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9e66ba-7358-4ba6-b72e-21eb67c6fbe3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:07.905804 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9e66ba-7358-4ba6-b72e-21eb67c6fbe3 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:07.907930 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:07.922346 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-cb9e66ba-7358-4ba6-b72e-21eb67c6fbe3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:07.922346 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:07.922346 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:07.922346 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:07.922346 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:07.922346 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:07.922346 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:07.922346 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:07.922346 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:07.922346 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:07.922346 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:07.922346 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:07.922346 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:33:07.922346 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:33:07.922346 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:33:07.923720 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:33:07.923720 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:07.923720 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:07.923720 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:07.923720 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:07.923720 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:07.923720 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:07.923720 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:07.923720 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:07.923720 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:07.923720 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:07.923720 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:07.923720 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:07.923720 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:07.923720 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:07.923720 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:07.923720 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:07.924589 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:33:07.924589 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:33:07.924589 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:33:07.924589 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:33:07.924589 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:33:07.924589 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:33:07.924589 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:07.924589 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:07.931003 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 258/536] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:33:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 414 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:07.934128 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e66ba-7358-4ba6-b72e-21eb67c6fbe3 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:07.945980 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a8aec2-65ab-4f6f-a008-88c04cab9f30 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:08.020444 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b525ed-b182-4dfd-9538-6cb072050196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:08.021034 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b525ed-b182-4dfd-9538-6cb072050196 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:08.021317 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b525ed-b182-4dfd-9538-6cb072050196 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:08.025144 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b525ed-b182-4dfd-9538-6cb072050196 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:08.033129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b525ed-b182-4dfd-9538-6cb072050196 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:08.034667 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b525ed-b182-4dfd-9538-6cb072050196 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:08.052963 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-17b525ed-b182-4dfd-9538-6cb072050196 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:08.052963 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:08.052963 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:08.052963 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:08.052963 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:08.052963 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:08.052963 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:08.052963 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:08.052963 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:08.052963 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:08.052963 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:08.052963 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:08.052963 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:33:08.052963 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:33:08.052963 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:33:08.053916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:33:08.053916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:08.053916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:08.053916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:08.053916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:08.053916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:08.053916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:08.053916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:08.053916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:08.053916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:08.053916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:08.053916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:08.053916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:08.053916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:08.053916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:08.053916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:08.053916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:08.055122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:33:08.055122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:33:08.055122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:33:08.055122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:33:08.055122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:33:08.055122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:33:08.055122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:08.055122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:08.055122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 279/537] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:33:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:08.061865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b525ed-b182-4dfd-9538-6cb072050196 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:08.176851 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e66ba-7358-4ba6-b72e-21eb67c6fbe3 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:08.230458 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea70974-3218-4090-8102-3495d4bbf0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:08.231133 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea70974-3218-4090-8102-3495d4bbf0b7 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:08.231426 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea70974-3218-4090-8102-3495d4bbf0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:08.231702 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea70974-3218-4090-8102-3495d4bbf0b7 None admin] PATH_INFO: `/v3/projects` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:08.232420 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea70974-3218-4090-8102-3495d4bbf0b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:08.232731 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea70974-3218-4090-8102-3495d4bbf0b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:08.234232 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea70974-3218-4090-8102-3495d4bbf0b7 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:08.263437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 259/538] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Sep 22 02:33:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:33:08.272815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea70974-3218-4090-8102-3495d4bbf0b7 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:08.294197 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:08.298024 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b525ed-b182-4dfd-9538-6cb072050196 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:08.339333 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ddfe9-cce6-44c3-b992-b5f8115b9b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:08.339999 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ddfe9-cce6-44c3-b992-b5f8115b9b54 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:08.340289 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ddfe9-cce6-44c3-b992-b5f8115b9b54 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:08.340571 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ddfe9-cce6-44c3-b992-b5f8115b9b54 None admin] PATH_INFO: `/v3/projects` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:08.341634 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ddfe9-cce6-44c3-b992-b5f8115b9b54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:08.341954 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ddfe9-cce6-44c3-b992-b5f8115b9b54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:08.343470 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ddfe9-cce6-44c3-b992-b5f8115b9b54 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:08.365187 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 280/539] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Sep 22 02:33:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:08.365986 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:08.372124 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ddfe9-cce6-44c3-b992-b5f8115b9b54 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:08.495091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea70974-3218-4090-8102-3495d4bbf0b7 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:08.552195 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2b9b0-cd9c-4994-b381-48366d87f2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:08.553005 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2b9b0-cd9c-4994-b381-48366d87f2b1 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:08.553393 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2b9b0-cd9c-4994-b381-48366d87f2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:08.553784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2b9b0-cd9c-4994-b381-48366d87f2b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:08.555064 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b2b9b0-cd9c-4994-b381-48366d87f2b1 None admin] RBAC: Adding query filter params (role.id=fb2fbee669e14ffbbe0aa70642eab1ce, scope.project.id=2a792f8dc45145d687ddd58c94d44b5e, user.id=a0f635200ad241b3b31ed49701bc511b) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:08.555540 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b2b9b0-cd9c-4994-b381-48366d87f2b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:08.558035 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b2b9b0-cd9c-4994-b381-48366d87f2b1 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:08.575863 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 260/540] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:33:08 2022] GET /identity/v3/role_assignments?user.id=a0f635200ad241b3b31ed49701bc511b&scope.project.id=2a792f8dc45145d687ddd58c94d44b5e&role.id=fb2fbee669e14ffbbe0aa70642eab1ce => generated 259 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:33:08.639393 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ddfe9-cce6-44c3-b992-b5f8115b9b54 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:08.712342 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53165c6b-d2cd-4b1e-a163-14bf3e8ad782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:08.713382 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53165c6b-d2cd-4b1e-a163-14bf3e8ad782 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:08.713879 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53165c6b-d2cd-4b1e-a163-14bf3e8ad782 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:08.714356 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53165c6b-d2cd-4b1e-a163-14bf3e8ad782 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:08.715913 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53165c6b-d2cd-4b1e-a163-14bf3e8ad782 None admin] RBAC: Adding query filter params (role.id=fb2fbee669e14ffbbe0aa70642eab1ce, scope.project.id=2a792f8dc45145d687ddd58c94d44b5e, user.id=a08a0ed98bc14328a3d24309d7996b51) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:08.716442 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53165c6b-d2cd-4b1e-a163-14bf3e8ad782 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:08.719202 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53165c6b-d2cd-4b1e-a163-14bf3e8ad782 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:08.734345 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 281/541] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:33:08 2022] GET /identity/v3/role_assignments?user.id=a08a0ed98bc14328a3d24309d7996b51&scope.project.id=2a792f8dc45145d687ddd58c94d44b5e&role.id=fb2fbee669e14ffbbe0aa70642eab1ce => generated 259 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:08.749554 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c3a5de-765b-4fed-b59c-e33abd607c5f None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:08.750040 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c3a5de-765b-4fed-b59c-e33abd607c5f None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:08.750505 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c3a5de-765b-4fed-b59c-e33abd607c5f None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:08.753477 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 261/542] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:33:08.760025 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d5775-a8fb-4090-a37f-4e8a78bc13c4 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:08.760661 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d5775-a8fb-4090-a37f-4e8a78bc13c4 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:08.761220 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d5775-a8fb-4090-a37f-4e8a78bc13c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:08.788986 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-0f1d5775-a8fb-4090-a37f-4e8a78bc13c4 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:08.790713 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-0f1d5775-a8fb-4090-a37f-4e8a78bc13c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:08.880573 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:08.909368 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 282/543] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:33:09.103661 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee841a-1ef7-4cdc-a635-e4801e237144 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:09.104677 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee841a-1ef7-4cdc-a635-e4801e237144 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:09.105185 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee841a-1ef7-4cdc-a635-e4801e237144 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:09.107162 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 262/544] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:33:09.113188 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3e71f-21a6-4ee4-917a-0cf4bd2c8b5a None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:09.113803 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3e71f-21a6-4ee4-917a-0cf4bd2c8b5a None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:09.114382 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3e71f-21a6-4ee4-917a-0cf4bd2c8b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:09.136920 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-26f3e71f-21a6-4ee4-917a-0cf4bd2c8b5a None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:09.138460 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-26f3e71f-21a6-4ee4-917a-0cf4bd2c8b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:09.239600 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 283/545] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:33:09.252417 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee841a-1ef7-4cdc-a635-e4801e237144 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:09.343307 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:09.373113 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee841a-1ef7-4cdc-a635-e4801e237144 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:09.415888 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40186242-51b9-418e-8949-b5527f994636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:09.416477 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40186242-51b9-418e-8949-b5527f994636 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:09.416757 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40186242-51b9-418e-8949-b5527f994636 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:09.417032 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40186242-51b9-418e-8949-b5527f994636 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:09.430441 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40186242-51b9-418e-8949-b5527f994636 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:33:09.436218 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-40186242-51b9-418e-8949-b5527f994636 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:33:09.436218 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:09.436218 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:09.436218 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:09.436218 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:09.436218 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:09.436218 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:09.436218 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:09.436218 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:09.436218 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:09.436218 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:09.436218 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:09.436218 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:33:09.436218 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:33:09.436218 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:33:09.436218 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:33:09.437168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:33:09.437168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:33:09.437168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:33:09.437168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:33:09.437168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:09.437168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:09.437168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:09.437168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:09.437168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:09.437168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:09.437168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:09.437168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:09.437168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:09.437168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:09.437168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:09.437168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:09.437974 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:09.437974 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:09.437974 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:09.437974 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:09.437974 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:33:09.437974 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:33:09.437974 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:33:09.437974 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:33:09.437974 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:33:09.437974 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:33:09.437974 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:33:09.437974 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:09.437974 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 263/546] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Sep 22 02:33:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:33:09.443664 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f3e71f-21a6-4ee4-917a-0cf4bd2c8b5a None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:09.602373 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f3e71f-21a6-4ee4-917a-0cf4bd2c8b5a None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:09.621815 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:09.642302 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b33afe-d5d7-48a2-a468-53eaca602a7a None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:09.642594 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b33afe-d5d7-48a2-a468-53eaca602a7a None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:09.642912 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b33afe-d5d7-48a2-a468-53eaca602a7a None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:09.644504 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 264/547] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:33:09.646449 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68812acb-e4c3-4821-95c1-de75cc93c6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:09.647198 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68812acb-e4c3-4821-95c1-de75cc93c6c8 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:09.647489 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68812acb-e4c3-4821-95c1-de75cc93c6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:09.647831 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68812acb-e4c3-4821-95c1-de75cc93c6c8 None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:09.648776 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68812acb-e4c3-4821-95c1-de75cc93c6c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:09.649160 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68812acb-e4c3-4821-95c1-de75cc93c6c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:09.650575 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68812acb-e4c3-4821-95c1-de75cc93c6c8 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:09.656550 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928926dd-1594-4ca6-84c0-4ccf85d7d844 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:09.657571 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928926dd-1594-4ca6-84c0-4ccf85d7d844 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:09.658323 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928926dd-1594-4ca6-84c0-4ccf85d7d844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:09.660664 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 284/548] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Sep 22 02:33:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:09.672046 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68812acb-e4c3-4821-95c1-de75cc93c6c8 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:09.706148 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-928926dd-1594-4ca6-84c0-4ccf85d7d844 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:09.708849 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-928926dd-1594-4ca6-84c0-4ccf85d7d844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:09.824473 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68812acb-e4c3-4821-95c1-de75cc93c6c8 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:09.867172 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 265/549] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:33:09.868496 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928926dd-1594-4ca6-84c0-4ccf85d7d844 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:09.878660 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f90f32-163d-4f0d-ae60-4642a5677c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:09.879592 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f90f32-163d-4f0d-ae60-4642a5677c94 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:09.880038 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f90f32-163d-4f0d-ae60-4642a5677c94 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:09.880470 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f90f32-163d-4f0d-ae60-4642a5677c94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:09.898229 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f90f32-163d-4f0d-ae60-4642a5677c94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:09.900318 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f90f32-163d-4f0d-ae60-4642a5677c94 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:09.921195 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-21f90f32-163d-4f0d-ae60-4642a5677c94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:09.921195 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:09.921195 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:09.921195 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:09.921195 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:09.921195 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:09.921195 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:09.921195 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:09.921195 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:09.921195 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:09.921195 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:09.921195 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:09.921195 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:09.921195 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:09.921195 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:09.921195 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:09.922881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:09.922881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:09.922881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:09.922881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:09.922881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:09.922881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:09.922881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:09.922881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:09.922881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:09.922881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:09.922881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:09.922881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:09.922881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:09.922881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:09.922881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:09.922881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:09.922881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:09.924427 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:09.924427 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:09.924427 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:09.931168 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 285/550] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:09.936906 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f90f32-163d-4f0d-ae60-4642a5677c94 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:09.994340 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:10.021584 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:10.103768 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928926dd-1594-4ca6-84c0-4ccf85d7d844 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:10.180115 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f90f32-163d-4f0d-ae60-4642a5677c94 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:10.246764 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91f312-5760-4ea3-8f0f-ea3c5e2b2ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:10.247461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91f312-5760-4ea3-8f0f-ea3c5e2b2ace None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:10.247817 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91f312-5760-4ea3-8f0f-ea3c5e2b2ace None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:10.248070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91f312-5760-4ea3-8f0f-ea3c5e2b2ace None admin] PATH_INFO: `/v3/roles/service` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:10.249694 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd81e1f2-0201-4447-b0ce-0d4bc26755b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:10.251154 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81e1f2-0201-4447-b0ce-0d4bc26755b5 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:10.251915 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81e1f2-0201-4447-b0ce-0d4bc26755b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:10.252693 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81e1f2-0201-4447-b0ce-0d4bc26755b5 None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:10.254176 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd81e1f2-0201-4447-b0ce-0d4bc26755b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:10.255053 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd81e1f2-0201-4447-b0ce-0d4bc26755b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:10.258221 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd81e1f2-0201-4447-b0ce-0d4bc26755b5 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:10.286109 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b91f312-5760-4ea3-8f0f-ea3c5e2b2ace None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:33:10.289644 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 286/551] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:33:10.295805 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd81e1f2-0201-4447-b0ce-0d4bc26755b5 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:10.299278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-1b91f312-5760-4ea3-8f0f-ea3c5e2b2ace None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:33:10.299278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:10.299278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:10.299278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:10.299278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:10.299278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:10.299278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:10.299278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:10.299278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:10.299278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:10.299278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:10.299278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:10.299278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:33:10.299278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:33:10.299278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:33:10.299278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:33:10.301195 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:33:10.301195 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:33:10.301195 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:33:10.301195 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:33:10.301195 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:10.301195 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:10.301195 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:10.301195 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:10.301195 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:10.301195 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:10.301195 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:10.301195 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:10.301195 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:10.301195 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:10.301195 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:10.301195 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:10.302980 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:10.302980 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:10.302980 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:10.302980 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:10.302980 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:33:10.302980 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:33:10.302980 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:33:10.302980 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:33:10.302980 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:33:10.302980 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:33:10.302980 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:33:10.302980 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:10.308184 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 266/552] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Sep 22 02:33:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 440 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:10.309951 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91f312-5760-4ea3-8f0f-ea3c5e2b2ace None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:10.437728 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:10.456552 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd81e1f2-0201-4447-b0ce-0d4bc26755b5 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:10.499465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f175c4-43f1-4ce1-9fcd-88b38e4832d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:10.500117 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f175c4-43f1-4ce1-9fcd-88b38e4832d1 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:10.500464 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f175c4-43f1-4ce1-9fcd-88b38e4832d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:10.500704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f175c4-43f1-4ce1-9fcd-88b38e4832d1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:10.510632 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f175c4-43f1-4ce1-9fcd-88b38e4832d1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:10.512171 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f175c4-43f1-4ce1-9fcd-88b38e4832d1 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:10.513868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91f312-5760-4ea3-8f0f-ea3c5e2b2ace None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:10.526028 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-82f175c4-43f1-4ce1-9fcd-88b38e4832d1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 02:33:10.526028 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:10.526028 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:10.526028 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:10.526028 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:10.526028 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:10.526028 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:10.526028 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:10.526028 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:10.526028 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:10.526028 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:10.526028 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:10.526028 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:33:10.526028 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:33:10.526028 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:33:10.526028 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:33:10.527786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:10.527786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:10.527786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:33:10.527786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:10.527786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:33:10.527786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:10.527786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:10.527786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:10.527786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:10.527786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:10.527786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:10.527786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:10.527786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:10.527786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:10.527786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:10.527786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:10.527786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:10.527786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:10.529556 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:10.529556 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:10.529556 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:33:10.529556 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:33:10.529556 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:33:10.529556 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:33:10.529556 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:33:10.529556 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:33:10.529556 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 02:33:10.529556 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:10.529556 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 287/553] 10.222.0.20 () {56 vars in 1093 bytes} [Thu Sep 22 02:33:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:10.529556 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f175c4-43f1-4ce1-9fcd-88b38e4832d1 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:10.565107 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:10.585858 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288fd920-7c05-4547-ae22-9cfd21de3b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:10.587528 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288fd920-7c05-4547-ae22-9cfd21de3b60 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:10.588295 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288fd920-7c05-4547-ae22-9cfd21de3b60 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:10.589064 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288fd920-7c05-4547-ae22-9cfd21de3b60 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:10.608429 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288fd920-7c05-4547-ae22-9cfd21de3b60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:33:10.619214 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-288fd920-7c05-4547-ae22-9cfd21de3b60 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:33:10.619214 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:10.619214 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:10.619214 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:10.619214 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:10.619214 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:10.619214 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:10.619214 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:10.619214 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:10.619214 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:10.619214 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:10.619214 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:10.619214 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:33:10.619214 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:33:10.619214 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:33:10.619214 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:33:10.620982 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:33:10.620982 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:33:10.620982 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:33:10.620982 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:33:10.620982 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:10.620982 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:10.620982 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:10.620982 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:10.620982 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:10.620982 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:10.620982 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:10.620982 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:10.620982 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:10.620982 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:10.620982 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:10.620982 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:10.622552 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:10.622552 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:10.622552 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:10.622552 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:10.622552 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:33:10.622552 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:33:10.622552 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:33:10.622552 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:33:10.622552 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:33:10.622552 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:33:10.622552 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:33:10.622552 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:10.627054 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 267/554] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Sep 22 02:33:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:10.628847 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288fd920-7c05-4547-ae22-9cfd21de3b60 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:10.657334 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f175c4-43f1-4ce1-9fcd-88b38e4832d1 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:10.701185 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf7dda9-cc57-42f2-8e63-987e82f4c5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:10.701710 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf7dda9-cc57-42f2-8e63-987e82f4c5f2 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:10.701993 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf7dda9-cc57-42f2-8e63-987e82f4c5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:10.702308 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf7dda9-cc57-42f2-8e63-987e82f4c5f2 None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:10.702968 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf7dda9-cc57-42f2-8e63-987e82f4c5f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:10.703274 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf7dda9-cc57-42f2-8e63-987e82f4c5f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:10.704677 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf7dda9-cc57-42f2-8e63-987e82f4c5f2 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:10.719156 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 288/555] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Sep 22 02:33:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:10.720420 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf7dda9-cc57-42f2-8e63-987e82f4c5f2 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:10.735905 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:10.829440 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288fd920-7c05-4547-ae22-9cfd21de3b60 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:10.850355 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf7dda9-cc57-42f2-8e63-987e82f4c5f2 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:10.892594 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a72b3-8a97-4c57-814d-a64a8dc5d945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:10.893225 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a72b3-8a97-4c57-814d-a64a8dc5d945 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:10.893507 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a72b3-8a97-4c57-814d-a64a8dc5d945 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:10.893785 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a72b3-8a97-4c57-814d-a64a8dc5d945 None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:10.894464 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a72b3-8a97-4c57-814d-a64a8dc5d945 None admin] RBAC: Adding query filter params (name=service) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:10.894854 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a72b3-8a97-4c57-814d-a64a8dc5d945 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:10.895763 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c32f0-08bd-400f-81d6-a51d433f02fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:10.896323 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a72b3-8a97-4c57-814d-a64a8dc5d945 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:10.897193 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c32f0-08bd-400f-81d6-a51d433f02fe None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:10.898684 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c32f0-08bd-400f-81d6-a51d433f02fe None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:10.899367 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c32f0-08bd-400f-81d6-a51d433f02fe None admin] PATH_INFO: `/v3/users` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:10.900754 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525c32f0-08bd-400f-81d6-a51d433f02fe None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:10.901474 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525c32f0-08bd-400f-81d6-a51d433f02fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:10.904518 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525c32f0-08bd-400f-81d6-a51d433f02fe None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:10.911012 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 289/556] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Sep 22 02:33:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:10.915520 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a72b3-8a97-4c57-814d-a64a8dc5d945 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:10.933051 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 268/557] 10.222.0.20 () {56 vars in 1133 bytes} [Thu Sep 22 02:33:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:10.933926 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c32f0-08bd-400f-81d6-a51d433f02fe None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:10.990892 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:11.048154 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:11.093241 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a72b3-8a97-4c57-814d-a64a8dc5d945 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:11.119315 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c32f0-08bd-400f-81d6-a51d433f02fe None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:11.159484 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b8c4d-8917-4926-a9e7-7fc7ec50096b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:11.161099 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b8c4d-8917-4926-a9e7-7fc7ec50096b None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:11.161646 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b8c4d-8917-4926-a9e7-7fc7ec50096b None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:11.162071 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b8c4d-8917-4926-a9e7-7fc7ec50096b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:11.175646 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0b8c4d-8917-4926-a9e7-7fc7ec50096b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:11.177285 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0b8c4d-8917-4926-a9e7-7fc7ec50096b None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:11.189779 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6a052c-f027-4276-bf58-6be13bf73a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:11.190895 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a052c-f027-4276-bf58-6be13bf73a4c None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:11.191384 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a052c-f027-4276-bf58-6be13bf73a4c None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:11.191875 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a052c-f027-4276-bf58-6be13bf73a4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:11.193792 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-2d0b8c4d-8917-4926-a9e7-7fc7ec50096b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:11.193792 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:11.193792 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:11.193792 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:11.193792 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:11.193792 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:11.193792 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:11.193792 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:11.193792 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:11.193792 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:11.193792 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:11.193792 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:11.193792 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:11.193792 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:11.193792 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:11.193792 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:11.194811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:11.194811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:11.194811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:11.194811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:11.194811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:11.194811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:11.194811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:11.194811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:11.194811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:11.194811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:11.194811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:11.194811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:11.194811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:11.194811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:11.194811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:11.194811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:11.194811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:11.195767 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:11.195767 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:11.195767 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:11.199829 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 290/558] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:11.200592 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b8c4d-8917-4926-a9e7-7fc7ec50096b None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:11.213227 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6a052c-f027-4276-bf58-6be13bf73a4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:11.216014 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6a052c-f027-4276-bf58-6be13bf73a4c None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:11.244016 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-ca6a052c-f027-4276-bf58-6be13bf73a4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:11.244016 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:11.244016 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:11.244016 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:11.244016 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:11.244016 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:11.244016 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:11.244016 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:11.244016 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:11.244016 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:11.244016 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:11.244016 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:11.244016 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:11.244016 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:11.244016 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:11.244016 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:11.245001 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:11.245001 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:11.245001 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:11.245001 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:11.245001 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:11.245001 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:11.245001 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:11.245001 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:11.245001 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:11.245001 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:11.245001 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:11.245001 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:11.245001 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:11.245001 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:11.245001 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:11.245001 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:11.245001 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:11.246012 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:11.246012 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:11.246012 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:11.246012 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 269/559] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:11.248154 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35343ac7-705a-4974-906f-6a9936d175cc None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:11.248657 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35343ac7-705a-4974-906f-6a9936d175cc None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:11.249090 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35343ac7-705a-4974-906f-6a9936d175cc None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:11.255458 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 270/560] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:33:11.255458 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35343ac7-705a-4974-906f-6a9936d175cc None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:11.338970 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b8c4d-8917-4926-a9e7-7fc7ec50096b None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:11.385337 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:11.391575 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:11.401871 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35343ac7-705a-4974-906f-6a9936d175cc None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:11.405666 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb288a-c572-434d-a9b4-f478285feb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:11.406754 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb288a-c572-434d-a9b4-f478285feb70 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:11.407263 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb288a-c572-434d-a9b4-f478285feb70 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:11.407760 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb288a-c572-434d-a9b4-f478285feb70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:11.428056 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb288a-c572-434d-a9b4-f478285feb70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:11.431462 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb288a-c572-434d-a9b4-f478285feb70 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:11.445189 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e987f-7107-4c70-96b0-8729d39a7c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:11.445834 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e987f-7107-4c70-96b0-8729d39a7c83 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:11.446118 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e987f-7107-4c70-96b0-8729d39a7c83 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:11.446395 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e987f-7107-4c70-96b0-8729d39a7c83 None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:11.447608 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e987f-7107-4c70-96b0-8729d39a7c83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:11.447923 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e987f-7107-4c70-96b0-8729d39a7c83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:11.449404 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e987f-7107-4c70-96b0-8729d39a7c83 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:11.459244 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-b2bb288a-c572-434d-a9b4-f478285feb70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:11.459244 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:11.459244 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:11.459244 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:11.459244 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:11.459244 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:11.459244 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:11.459244 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:11.459244 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:11.459244 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:11.459244 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:11.459244 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:11.459244 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:11.459244 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:11.459244 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:11.459244 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:11.460260 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:11.460260 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:11.460260 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:11.460260 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:11.460260 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:11.460260 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:11.460260 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:11.460260 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:11.460260 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:11.460260 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:11.460260 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:11.460260 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:11.460260 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:11.460260 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:11.460260 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:11.460260 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:11.460260 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:11.461263 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:11.461263 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:11.461263 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:11.466526 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 291/561] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:11.466526 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb288a-c572-434d-a9b4-f478285feb70 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:11.477125 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 271/562] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:11.478928 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bbeec-cbee-48ca-9612-b28e594fb525 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:11.479257 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bbeec-cbee-48ca-9612-b28e594fb525 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:11.479442 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bbeec-cbee-48ca-9612-b28e594fb525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:11.499749 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-250bbeec-cbee-48ca-9612-b28e594fb525 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:11.500963 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-250bbeec-cbee-48ca-9612-b28e594fb525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:11.574302 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 272/563] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:33:11.575209 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250bbeec-cbee-48ca-9612-b28e594fb525 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:11.604602 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb288a-c572-434d-a9b4-f478285feb70 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:11.649761 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48cc8c0-b3c9-4508-8ae9-82db9a9478b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:11.650677 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48cc8c0-b3c9-4508-8ae9-82db9a9478b9 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:11.651007 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48cc8c0-b3c9-4508-8ae9-82db9a9478b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:11.651286 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48cc8c0-b3c9-4508-8ae9-82db9a9478b9 None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:11.652061 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48cc8c0-b3c9-4508-8ae9-82db9a9478b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:11.652390 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48cc8c0-b3c9-4508-8ae9-82db9a9478b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:11.653928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48cc8c0-b3c9-4508-8ae9-82db9a9478b9 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:11.675617 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 292/564] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:11.675617 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54952015-8b0d-4ef8-a0e9-53e61544da8d None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:11.675617 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54952015-8b0d-4ef8-a0e9-53e61544da8d None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:11.675617 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54952015-8b0d-4ef8-a0e9-53e61544da8d None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:11.675617 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 293/565] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:33:11.675617 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54952015-8b0d-4ef8-a0e9-53e61544da8d None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:11.693599 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:11.717631 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250bbeec-cbee-48ca-9612-b28e594fb525 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:11.726578 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:11.765839 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4820e0e4-23ca-4989-a753-0b5b147ab963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:11.766577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820e0e4-23ca-4989-a753-0b5b147ab963 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:11.767091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820e0e4-23ca-4989-a753-0b5b147ab963 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:11.767502 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820e0e4-23ca-4989-a753-0b5b147ab963 None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:11.769607 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4820e0e4-23ca-4989-a753-0b5b147ab963 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:11.770078 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4820e0e4-23ca-4989-a753-0b5b147ab963 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:11.771704 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4820e0e4-23ca-4989-a753-0b5b147ab963 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:11.789068 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 273/566] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:11.790108 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4820e0e4-23ca-4989-a753-0b5b147ab963 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:11.810160 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54952015-8b0d-4ef8-a0e9-53e61544da8d None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:11.850149 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64637cc9-5d61-4c61-8686-87db710d31d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:11.850869 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64637cc9-5d61-4c61-8686-87db710d31d7 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:11.851665 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64637cc9-5d61-4c61-8686-87db710d31d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:11.851978 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64637cc9-5d61-4c61-8686-87db710d31d7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:11.859847 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64637cc9-5d61-4c61-8686-87db710d31d7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:11.861321 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64637cc9-5d61-4c61-8686-87db710d31d7 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:11.880936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-64637cc9-5d61-4c61-8686-87db710d31d7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 22 02:33:11.880936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:11.880936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:11.880936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:11.880936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:11.880936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:11.880936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:11.880936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:11.880936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:11.880936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:11.880936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:11.880936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:11.880936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:33:11.880936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:33:11.880936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:33:11.880936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:33:11.882058 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:11.882058 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:11.882058 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:33:11.882058 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:11.882058 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:33:11.882058 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:11.882058 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:11.882058 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:11.882058 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:11.882058 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:11.882058 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:11.882058 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:11.882058 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:11.882058 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:11.882058 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:11.882058 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:11.882058 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:11.882058 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:11.883045 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:11.883045 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:11.883045 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:33:11.883045 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:33:11.883045 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:33:11.883045 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:33:11.883045 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:33:11.883045 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:33:11.883045 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 22 02:33:11.883045 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:11.883045 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 294/567] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Sep 22 02:33:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:11.884518 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac4523-b581-411f-a9a4-d3a8e17bc0aa None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:11.884857 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac4523-b581-411f-a9a4-d3a8e17bc0aa None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:11.885184 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac4523-b581-411f-a9a4-d3a8e17bc0aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:11.905462 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-76ac4523-b581-411f-a9a4-d3a8e17bc0aa None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:11.906696 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-76ac4523-b581-411f-a9a4-d3a8e17bc0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:11.923892 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4820e0e4-23ca-4989-a753-0b5b147ab963 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:11.959787 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e7fb8-e861-4a8d-be88-f17980da0f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:11.960974 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e7fb8-e861-4a8d-be88-f17980da0f3a None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:11.961253 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e7fb8-e861-4a8d-be88-f17980da0f3a None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:11.961524 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e7fb8-e861-4a8d-be88-f17980da0f3a None admin] PATH_INFO: `/v3/users/glance` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:11.969858 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60e7fb8-e861-4a8d-be88-f17980da0f3a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:11.971377 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60e7fb8-e861-4a8d-be88-f17980da0f3a None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:11.983635 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 295/568] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:33:11.983877 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-c60e7fb8-e861-4a8d-be88-f17980da0f3a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 22 02:33:11.983877 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:11.983877 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:11.983877 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:11.983877 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:11.983877 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:11.983877 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:11.983877 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:11.983877 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:11.983877 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:11.983877 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:11.983877 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:11.983877 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:33:11.983877 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:33:11.983877 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:33:11.983877 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:33:11.985562 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:11.985562 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:11.985562 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:33:11.985562 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:11.985562 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:33:11.985562 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:11.985562 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:11.985562 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:11.985562 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:11.985562 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:11.985562 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:11.985562 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:11.985562 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:11.985562 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:11.985562 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:11.985562 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:11.985562 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:11.985562 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:11.987446 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:11.987446 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:11.987446 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:33:11.987446 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:33:11.987446 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:33:11.987446 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:33:11.987446 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:33:11.987446 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:33:11.987446 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 22 02:33:11.987446 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:11.988454 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac4523-b581-411f-a9a4-d3a8e17bc0aa None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:11.988590 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 274/569] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:33:11 2022] GET /identity/v3/users/glance => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:11.988590 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e7fb8-e861-4a8d-be88-f17980da0f3a None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:11.996733 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:12.026886 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:12.133258 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac4523-b581-411f-a9a4-d3a8e17bc0aa None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:12.143975 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e7fb8-e861-4a8d-be88-f17980da0f3a None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:12.181384 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef75753-8f7a-4dc3-8c44-e10c8ae6bebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:12.182030 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef75753-8f7a-4dc3-8c44-e10c8ae6bebc None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:12.182317 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef75753-8f7a-4dc3-8c44-e10c8ae6bebc None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:12.182590 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef75753-8f7a-4dc3-8c44-e10c8ae6bebc None admin] PATH_INFO: `/v3/roles/service` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:12.194118 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef75753-8f7a-4dc3-8c44-e10c8ae6bebc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:33:12.196057 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfd7cf8-ed48-47ca-beae-ab427efe9465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:12.197286 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd7cf8-ed48-47ca-beae-ab427efe9465 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:12.197885 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd7cf8-ed48-47ca-beae-ab427efe9465 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:12.198487 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd7cf8-ed48-47ca-beae-ab427efe9465 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:12.200136 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-8ef75753-8f7a-4dc3-8c44-e10c8ae6bebc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:33:12.200136 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:12.200136 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:12.200136 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:12.200136 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:12.200136 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:12.200136 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:12.200136 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:12.200136 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:12.200136 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:12.200136 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:12.200136 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:12.200136 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:33:12.200136 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:33:12.200136 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:33:12.200136 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:33:12.201150 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:33:12.201150 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:33:12.201150 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:33:12.201150 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:33:12.201150 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:12.201150 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:12.201150 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:12.201150 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:12.201150 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:12.201150 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:12.201150 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:12.201150 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:12.201150 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:12.201150 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:12.201150 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:12.201150 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:12.202104 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:12.202104 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:12.202104 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:12.202104 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:12.202104 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:33:12.202104 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:33:12.202104 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:33:12.202104 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:33:12.202104 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:33:12.202104 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:33:12.202104 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:33:12.202104 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:12.206535 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 275/570] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Sep 22 02:33:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:12.207277 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef75753-8f7a-4dc3-8c44-e10c8ae6bebc None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:12.215496 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfd7cf8-ed48-47ca-beae-ab427efe9465 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:12.218507 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfd7cf8-ed48-47ca-beae-ab427efe9465 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:12.241819 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-ddfd7cf8-ed48-47ca-beae-ab427efe9465 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:12.241819 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:12.241819 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:12.241819 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:12.241819 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:12.241819 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:12.241819 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:12.241819 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:12.241819 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:12.241819 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:12.241819 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:12.241819 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:12.241819 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:33:12.241819 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:33:12.241819 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:33:12.243052 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:33:12.243052 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:12.243052 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:12.243052 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:12.243052 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:12.243052 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:12.243052 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:12.243052 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:12.243052 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:12.243052 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:12.243052 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:12.243052 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:12.243052 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:12.243052 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:12.243052 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:12.243052 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:12.243052 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:12.244068 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:33:12.244068 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:33:12.244068 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:33:12.244068 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:33:12.244068 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:33:12.244068 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:33:12.244068 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:12.244068 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:12.251705 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 296/571] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:33:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:12.251705 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfd7cf8-ed48-47ca-beae-ab427efe9465 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:12.278357 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:12.325245 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef75753-8f7a-4dc3-8c44-e10c8ae6bebc None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:12.366199 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b25c2-de95-495c-a244-5111c96ad879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:12.367336 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b25c2-de95-495c-a244-5111c96ad879 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:12.367614 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b25c2-de95-495c-a244-5111c96ad879 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:12.367886 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b25c2-de95-495c-a244-5111c96ad879 None admin] PATH_INFO: `/v3/users` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:12.368780 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81b25c2-de95-495c-a244-5111c96ad879 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:12.368894 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:12.368985 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81b25c2-de95-495c-a244-5111c96ad879 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:12.371030 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81b25c2-de95-495c-a244-5111c96ad879 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:12.386977 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfd7cf8-ed48-47ca-beae-ab427efe9465 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:12.390562 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 276/572] 10.222.0.20 () {56 vars in 1135 bytes} [Thu Sep 22 02:33:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:12.391681 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b25c2-de95-495c-a244-5111c96ad879 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:12.423025 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ece4ab-ae1f-4af0-8c9c-ed7eee15f508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:12.423762 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ece4ab-ae1f-4af0-8c9c-ed7eee15f508 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:12.423933 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ece4ab-ae1f-4af0-8c9c-ed7eee15f508 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:12.424177 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ece4ab-ae1f-4af0-8c9c-ed7eee15f508 None admin] PATH_INFO: `/v3/users` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:12.424926 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ece4ab-ae1f-4af0-8c9c-ed7eee15f508 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:12.425218 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ece4ab-ae1f-4af0-8c9c-ed7eee15f508 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:12.426682 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ece4ab-ae1f-4af0-8c9c-ed7eee15f508 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:12.448208 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 297/573] 10.222.0.20 () {56 vars in 1137 bytes} [Thu Sep 22 02:33:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:12.449061 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ece4ab-ae1f-4af0-8c9c-ed7eee15f508 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:12.527036 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b25c2-de95-495c-a244-5111c96ad879 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:12.557350 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:12.571262 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ff6f0-a387-46d6-b062-25f16833d334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:12.572007 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ff6f0-a387-46d6-b062-25f16833d334 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:12.572398 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ff6f0-a387-46d6-b062-25f16833d334 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:12.572778 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ff6f0-a387-46d6-b062-25f16833d334 None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:12.573702 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6ff6f0-a387-46d6-b062-25f16833d334 None admin] RBAC: Adding query filter params (name=service) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:12.574161 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6ff6f0-a387-46d6-b062-25f16833d334 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:12.575823 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6ff6f0-a387-46d6-b062-25f16833d334 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:12.585685 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 277/574] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Sep 22 02:33:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:12.586549 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ff6f0-a387-46d6-b062-25f16833d334 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:12.586798 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ece4ab-ae1f-4af0-8c9c-ed7eee15f508 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:12.625323 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a605f6-dc7e-4391-82d7-796b25d3b3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:12.626030 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a605f6-dc7e-4391-82d7-796b25d3b3f9 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:12.626473 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a605f6-dc7e-4391-82d7-796b25d3b3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:12.626855 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a605f6-dc7e-4391-82d7-796b25d3b3f9 None admin] PATH_INFO: `/v3/projects` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:12.627765 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a605f6-dc7e-4391-82d7-796b25d3b3f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:12.628196 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a605f6-dc7e-4391-82d7-796b25d3b3f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:12.629803 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a605f6-dc7e-4391-82d7-796b25d3b3f9 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:12.642117 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 298/575] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Sep 22 02:33:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:12.643115 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a605f6-dc7e-4391-82d7-796b25d3b3f9 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:12.648207 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:12.705931 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ff6f0-a387-46d6-b062-25f16833d334 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:12.746203 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da66247-1dbd-438f-b1e1-abe7a3020fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:12.747037 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da66247-1dbd-438f-b1e1-abe7a3020fd1 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:12.747449 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da66247-1dbd-438f-b1e1-abe7a3020fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:12.747731 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da66247-1dbd-438f-b1e1-abe7a3020fd1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:12.756101 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a605f6-dc7e-4391-82d7-796b25d3b3f9 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:12.760587 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da66247-1dbd-438f-b1e1-abe7a3020fd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:33:12.766091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-0da66247-1dbd-438f-b1e1-abe7a3020fd1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:33:12.766091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:12.766091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:12.766091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:12.766091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:12.766091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:12.766091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:12.766091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:12.766091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:12.766091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:12.766091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:12.766091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:12.766091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:33:12.766091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:33:12.766091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:33:12.766091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:33:12.767913 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:33:12.767913 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:33:12.767913 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:33:12.767913 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:33:12.767913 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:12.767913 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:12.767913 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:12.767913 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:12.767913 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:12.767913 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:12.767913 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:12.767913 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:12.767913 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:12.767913 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:12.767913 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:12.767913 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:12.769682 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:12.769682 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:12.769682 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:12.769682 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:12.769682 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:33:12.769682 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:33:12.769682 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:33:12.769682 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:33:12.769682 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:33:12.769682 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:33:12.769682 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:33:12.769682 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:12.769682 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 278/576] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Sep 22 02:33:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:12.769682 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da66247-1dbd-438f-b1e1-abe7a3020fd1 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:12.803042 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208da2e9-ffc3-4f64-a68c-c089aca8d771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:12.803670 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208da2e9-ffc3-4f64-a68c-c089aca8d771 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:12.803953 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208da2e9-ffc3-4f64-a68c-c089aca8d771 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:12.804229 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208da2e9-ffc3-4f64-a68c-c089aca8d771 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:12.821530 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:12.883409 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208da2e9-ffc3-4f64-a68c-c089aca8d771 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:12.884874 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208da2e9-ffc3-4f64-a68c-c089aca8d771 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:12.887052 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da66247-1dbd-438f-b1e1-abe7a3020fd1 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:12.887776 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:12.926812 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364e3f4-173b-4479-928a-be63195664bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:12.927508 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364e3f4-173b-4479-928a-be63195664bc None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:12.927784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364e3f4-173b-4479-928a-be63195664bc None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:12.928060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364e3f4-173b-4479-928a-be63195664bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:12.939596 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364e3f4-173b-4479-928a-be63195664bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:12.941114 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364e3f4-173b-4479-928a-be63195664bc None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:12.956962 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-2364e3f4-173b-4479-928a-be63195664bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:12.956962 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:12.956962 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:12.956962 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:12.956962 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:12.956962 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:12.956962 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:12.956962 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:12.956962 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:12.956962 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:12.956962 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:12.956962 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:12.956962 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:12.956962 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:12.956962 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:12.956962 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:12.958898 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:12.958898 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:12.958898 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:12.958898 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:12.958898 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:12.958898 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:12.958898 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:12.958898 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:12.958898 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:12.958898 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:12.958898 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:12.958898 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:12.958898 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:12.958898 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:12.958898 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:12.958898 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:12.958898 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:12.961437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:12.961437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:12.961437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:12.961437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 279/577] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:12.961437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364e3f4-173b-4479-928a-be63195664bc None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:12.998242 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 299/578] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:33:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 02:33:12.999228 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208da2e9-ffc3-4f64-a68c-c089aca8d771 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:13.066760 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:13.075177 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364e3f4-173b-4479-928a-be63195664bc None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:13.114524 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c2b77-26b1-4e5f-bd42-485f4f14d0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:13.115446 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208da2e9-ffc3-4f64-a68c-c089aca8d771 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:13.115565 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c2b77-26b1-4e5f-bd42-485f4f14d0c5 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:13.116023 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c2b77-26b1-4e5f-bd42-485f4f14d0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:13.116559 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c2b77-26b1-4e5f-bd42-485f4f14d0c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:13.121149 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:13.135847 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c2b77-26b1-4e5f-bd42-485f4f14d0c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:13.137570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c2b77-26b1-4e5f-bd42-485f4f14d0c5 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:13.155473 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-e62c2b77-26b1-4e5f-bd42-485f4f14d0c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:13.155473 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:13.155473 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:13.155473 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:13.155473 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:13.155473 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:13.155473 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:13.155473 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:13.155473 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:13.155473 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:13.155473 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:13.155473 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:13.155473 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:13.155473 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:13.155473 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:13.155473 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:13.158456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:13.158456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:13.158456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:13.158456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:13.158456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:13.158456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:13.158456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:13.158456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:13.158456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:13.158456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:13.158456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:13.158456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:13.158456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:13.158456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:13.158456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:13.158456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:13.158456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:13.160338 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:13.160338 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:13.160338 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:13.160338 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 280/579] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:13.160338 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c2b77-26b1-4e5f-bd42-485f4f14d0c5 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:13.173126 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a666e-cbae-4fd2-82dd-316314d533fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:13.173967 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a666e-cbae-4fd2-82dd-316314d533fe None admin] REQUEST_METHOD: `PUT` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:13.174110 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a666e-cbae-4fd2-82dd-316314d533fe None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:13.174346 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a666e-cbae-4fd2-82dd-316314d533fe None admin] PATH_INFO: `/v3/projects/2a792f8dc45145d687ddd58c94d44b5e/users/a64ed6c8e15d479887ab1dc59997d715/roles/fb2fbee669e14ffbbe0aa70642eab1ce` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:13.194392 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880a666e-cbae-4fd2-82dd-316314d533fe None admin] RBAC: Authorizing `identity:create_grant(project_id=2a792f8dc45145d687ddd58c94d44b5e, user_id=a64ed6c8e15d479887ab1dc59997d715, role_id=fb2fbee669e14ffbbe0aa70642eab1ce)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:13.196104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880a666e-cbae-4fd2-82dd-316314d533fe None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:13.209879 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 300/580] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Sep 22 02:33:12 2022] PUT /identity/v3/projects/2a792f8dc45145d687ddd58c94d44b5e/users/a64ed6c8e15d479887ab1dc59997d715/roles/fb2fbee669e14ffbbe0aa70642eab1ce => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 02:33:13.211488 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a666e-cbae-4fd2-82dd-316314d533fe None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:13.307981 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c2b77-26b1-4e5f-bd42-485f4f14d0c5 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:13.353162 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21394c0d-2677-485a-8fe4-fd0155ce8fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:13.354050 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21394c0d-2677-485a-8fe4-fd0155ce8fac None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:13.354526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21394c0d-2677-485a-8fe4-fd0155ce8fac None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:13.355006 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21394c0d-2677-485a-8fe4-fd0155ce8fac None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:13.355836 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21394c0d-2677-485a-8fe4-fd0155ce8fac None admin] RBAC: Adding query filter params (name=service) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:13.356236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21394c0d-2677-485a-8fe4-fd0155ce8fac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:13.357917 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21394c0d-2677-485a-8fe4-fd0155ce8fac None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:13.368482 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 281/581] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Sep 22 02:33:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:13.368790 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a666e-cbae-4fd2-82dd-316314d533fe None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:13.369362 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21394c0d-2677-485a-8fe4-fd0155ce8fac None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:13.391076 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:13.437526 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28b64f-d110-467c-81ac-1b97765c9934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:13.438581 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28b64f-d110-467c-81ac-1b97765c9934 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:13.439095 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28b64f-d110-467c-81ac-1b97765c9934 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:13.439383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28b64f-d110-467c-81ac-1b97765c9934 None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:13.440553 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f28b64f-d110-467c-81ac-1b97765c9934 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:13.441107 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f28b64f-d110-467c-81ac-1b97765c9934 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:13.443756 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f28b64f-d110-467c-81ac-1b97765c9934 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:13.458241 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 301/582] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:13.459277 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28b64f-d110-467c-81ac-1b97765c9934 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:13.480167 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:13.516318 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21394c0d-2677-485a-8fe4-fd0155ce8fac None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:13.579690 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2356b123-13ab-4796-a65f-6f502c2c2440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:13.580706 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356b123-13ab-4796-a65f-6f502c2c2440 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:13.581200 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356b123-13ab-4796-a65f-6f502c2c2440 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:13.581801 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356b123-13ab-4796-a65f-6f502c2c2440 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:13.596089 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28b64f-d110-467c-81ac-1b97765c9934 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:13.603306 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2356b123-13ab-4796-a65f-6f502c2c2440 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:13.605609 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2356b123-13ab-4796-a65f-6f502c2c2440 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:13.622467 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-2356b123-13ab-4796-a65f-6f502c2c2440 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:13.622467 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:13.622467 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:13.622467 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:13.622467 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:13.622467 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:13.622467 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:13.622467 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:13.622467 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:13.622467 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:13.622467 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:13.622467 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:13.622467 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:13.622467 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:13.622467 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:13.622467 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:13.623833 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:13.623833 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:13.623833 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:13.623833 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:13.623833 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:13.623833 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:13.623833 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:13.623833 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:13.623833 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:13.623833 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:13.623833 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:13.623833 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:13.623833 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:13.623833 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:13.623833 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:13.623833 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:13.623833 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:13.625189 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:13.625189 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:13.625189 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:13.625189 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 282/583] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:13.625189 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2356b123-13ab-4796-a65f-6f502c2c2440 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:13.644479 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cfce3c-773a-4f3f-86f9-dfe2746f7ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:13.645493 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cfce3c-773a-4f3f-86f9-dfe2746f7ba2 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:13.645985 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cfce3c-773a-4f3f-86f9-dfe2746f7ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:13.646479 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cfce3c-773a-4f3f-86f9-dfe2746f7ba2 None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:13.647366 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cfce3c-773a-4f3f-86f9-dfe2746f7ba2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:13.648596 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cfce3c-773a-4f3f-86f9-dfe2746f7ba2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:13.650317 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cfce3c-773a-4f3f-86f9-dfe2746f7ba2 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:13.670893 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 302/584] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:13.670893 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cfce3c-773a-4f3f-86f9-dfe2746f7ba2 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:13.706065 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:13.761446 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2356b123-13ab-4796-a65f-6f502c2c2440 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:13.781852 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:13.809852 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f76e56-ea6c-461e-9b25-2fb8be468905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:13.810603 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f76e56-ea6c-461e-9b25-2fb8be468905 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:13.810937 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f76e56-ea6c-461e-9b25-2fb8be468905 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:13.811209 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f76e56-ea6c-461e-9b25-2fb8be468905 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:13.823733 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f76e56-ea6c-461e-9b25-2fb8be468905 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:13.825208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f76e56-ea6c-461e-9b25-2fb8be468905 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:13.827213 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cfce3c-773a-4f3f-86f9-dfe2746f7ba2 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:13.841278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-90f76e56-ea6c-461e-9b25-2fb8be468905 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:13.841278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:13.841278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:13.841278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:13.841278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:13.841278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:13.841278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:13.841278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:13.841278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:13.841278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:13.841278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:13.841278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:13.841278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:13.841278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:13.841278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:13.841278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:13.843098 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:13.843098 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:13.843098 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:13.843098 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:13.843098 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:13.843098 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:13.843098 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:13.843098 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:13.843098 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:13.843098 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:13.843098 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:13.843098 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:13.843098 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:13.843098 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:13.843098 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:13.843098 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:13.843098 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:13.845394 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:13.845394 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:13.845394 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:13.845394 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 283/585] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:13.845394 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f76e56-ea6c-461e-9b25-2fb8be468905 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:13.890147 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bef72c-2df6-4e4b-b849-a70803e2de95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:13.891053 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bef72c-2df6-4e4b-b849-a70803e2de95 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:13.891529 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bef72c-2df6-4e4b-b849-a70803e2de95 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:13.892021 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bef72c-2df6-4e4b-b849-a70803e2de95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:13.906364 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bef72c-2df6-4e4b-b849-a70803e2de95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:13.908944 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bef72c-2df6-4e4b-b849-a70803e2de95 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:13.930744 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-e2bef72c-2df6-4e4b-b849-a70803e2de95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:13.930744 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:13.930744 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:13.930744 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:13.930744 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:13.930744 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:13.930744 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:13.930744 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:13.930744 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:13.930744 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:13.930744 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:13.930744 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:13.930744 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:33:13.930744 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:33:13.930744 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:33:13.931865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:33:13.931865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:13.931865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:13.931865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:13.931865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:13.931865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:13.931865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:13.931865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:13.931865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:13.931865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:13.931865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:13.931865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:13.931865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:13.931865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:13.931865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:13.931865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:13.931865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:13.932929 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:33:13.932929 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:33:13.932929 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:33:13.932929 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:33:13.932929 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:33:13.932929 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:33:13.932929 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:13.932929 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:13.933441 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 303/586] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:33:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:13.934604 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bef72c-2df6-4e4b-b849-a70803e2de95 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:13.983671 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f76e56-ea6c-461e-9b25-2fb8be468905 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:14.000604 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:14.034442 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf2107-e354-46f4-bad0-54a0c1171d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:14.035205 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf2107-e354-46f4-bad0-54a0c1171d4a None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:14.035553 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf2107-e354-46f4-bad0-54a0c1171d4a None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:14.035896 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf2107-e354-46f4-bad0-54a0c1171d4a None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:14.036616 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf2107-e354-46f4-bad0-54a0c1171d4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:14.036983 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf2107-e354-46f4-bad0-54a0c1171d4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:14.038587 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf2107-e354-46f4-bad0-54a0c1171d4a None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:14.052389 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 284/587] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:14.053094 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf2107-e354-46f4-bad0-54a0c1171d4a None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:14.115505 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bef72c-2df6-4e4b-b849-a70803e2de95 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:14.178647 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eebb80-5b53-4cb4-859c-6c3965e5ea7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:14.180243 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eebb80-5b53-4cb4-859c-6c3965e5ea7f None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:14.180874 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eebb80-5b53-4cb4-859c-6c3965e5ea7f None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:14.181377 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eebb80-5b53-4cb4-859c-6c3965e5ea7f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:14.184749 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:14.197494 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eebb80-5b53-4cb4-859c-6c3965e5ea7f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:14.200097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eebb80-5b53-4cb4-859c-6c3965e5ea7f None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:14.223169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-b0eebb80-5b53-4cb4-859c-6c3965e5ea7f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 22 02:33:14.223169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:14.223169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:14.223169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:14.223169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:14.223169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:14.223169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:14.223169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:14.223169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:14.223169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:14.223169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:14.223169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:14.223169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:33:14.223169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:33:14.223169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:33:14.223169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:33:14.224563 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:14.224563 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:14.224563 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:33:14.224563 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:14.224563 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:33:14.224563 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:14.224563 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:14.224563 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:14.224563 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:14.224563 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:14.224563 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:14.224563 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:14.224563 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:14.224563 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:14.224563 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:14.224563 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:14.224563 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:14.224563 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:14.225942 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:14.225942 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:14.225942 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:33:14.225942 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:33:14.225942 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:33:14.225942 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:33:14.225942 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:33:14.225942 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:33:14.225942 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 22 02:33:14.225942 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:14.226717 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf2107-e354-46f4-bad0-54a0c1171d4a None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:14.231204 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 304/588] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Sep 22 02:33:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:14.232605 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eebb80-5b53-4cb4-859c-6c3965e5ea7f None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:14.270098 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332390e-9d68-4ada-958e-81db211f94fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:14.270861 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332390e-9d68-4ada-958e-81db211f94fa None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:14.271059 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332390e-9d68-4ada-958e-81db211f94fa None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:14.271563 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332390e-9d68-4ada-958e-81db211f94fa None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:14.271958 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332390e-9d68-4ada-958e-81db211f94fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:14.272264 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332390e-9d68-4ada-958e-81db211f94fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:14.274016 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332390e-9d68-4ada-958e-81db211f94fa None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:14.291922 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 285/589] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:14.291922 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332390e-9d68-4ada-958e-81db211f94fa None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:14.332261 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:14.394749 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eebb80-5b53-4cb4-859c-6c3965e5ea7f None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:14.422093 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332390e-9d68-4ada-958e-81db211f94fa None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:14.436659 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f95993-0c94-4f41-9a36-de649ec98698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:14.437430 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f95993-0c94-4f41-9a36-de649ec98698 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:14.437939 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f95993-0c94-4f41-9a36-de649ec98698 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:14.438443 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f95993-0c94-4f41-9a36-de649ec98698 None admin] PATH_INFO: `/v3/projects` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:14.439816 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f95993-0c94-4f41-9a36-de649ec98698 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:14.440384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f95993-0c94-4f41-9a36-de649ec98698 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:14.443019 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f95993-0c94-4f41-9a36-de649ec98698 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:14.468507 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 305/590] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Sep 22 02:33:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:14.472236 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f95993-0c94-4f41-9a36-de649ec98698 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:14.473647 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c51fcc-8626-4dc2-a68b-333dce192f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:14.474738 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c51fcc-8626-4dc2-a68b-333dce192f00 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:14.475273 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c51fcc-8626-4dc2-a68b-333dce192f00 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:14.476266 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c51fcc-8626-4dc2-a68b-333dce192f00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:14.488240 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c51fcc-8626-4dc2-a68b-333dce192f00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:14.489917 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c51fcc-8626-4dc2-a68b-333dce192f00 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:14.507726 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-85c51fcc-8626-4dc2-a68b-333dce192f00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:14.507726 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:14.507726 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:14.507726 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:14.507726 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:14.507726 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:14.507726 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:14.507726 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:14.507726 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:14.507726 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:14.507726 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:14.507726 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:14.507726 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:33:14.507726 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:33:14.507726 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:33:14.508882 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:33:14.508882 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:14.508882 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:14.508882 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:14.508882 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:14.508882 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:14.508882 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:14.508882 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:14.508882 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:14.508882 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:14.508882 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:14.508882 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:14.508882 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:14.508882 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:14.508882 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:14.508882 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:14.508882 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:14.509992 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:33:14.509992 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:33:14.509992 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:33:14.509992 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:33:14.509992 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:33:14.509992 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:33:14.509992 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:14.509992 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:14.514110 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 286/591] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:33:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:14.515119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c51fcc-8626-4dc2-a68b-333dce192f00 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:14.558038 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:14.626155 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f95993-0c94-4f41-9a36-de649ec98698 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:14.636272 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:14.643745 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c51fcc-8626-4dc2-a68b-333dce192f00 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:14.665967 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d1933-3293-471f-813b-ebd6fddaed0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:14.666648 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d1933-3293-471f-813b-ebd6fddaed0e None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:14.667004 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d1933-3293-471f-813b-ebd6fddaed0e None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:14.667335 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d1933-3293-471f-813b-ebd6fddaed0e None admin] PATH_INFO: `/v3/users` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:14.668709 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d1933-3293-471f-813b-ebd6fddaed0e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:14.669184 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d1933-3293-471f-813b-ebd6fddaed0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:14.671002 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d1933-3293-471f-813b-ebd6fddaed0e None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:14.682933 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac48197-19e2-41d7-b6c1-0b34cd525c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:14.683580 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac48197-19e2-41d7-b6c1-0b34cd525c45 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:14.683865 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac48197-19e2-41d7-b6c1-0b34cd525c45 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:14.684287 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac48197-19e2-41d7-b6c1-0b34cd525c45 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:14.697101 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 306/592] 10.222.0.20 () {56 vars in 1139 bytes} [Thu Sep 22 02:33:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:14.697101 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d1933-3293-471f-813b-ebd6fddaed0e None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:14.699246 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac48197-19e2-41d7-b6c1-0b34cd525c45 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:14.700764 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac48197-19e2-41d7-b6c1-0b34cd525c45 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:14.713923 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-eac48197-19e2-41d7-b6c1-0b34cd525c45 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 02:33:14.713923 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:14.713923 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:14.713923 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:14.713923 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:14.713923 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:14.713923 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:14.713923 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:14.713923 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:14.713923 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:14.713923 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:14.713923 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:14.713923 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:33:14.713923 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:33:14.713923 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:33:14.713923 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:33:14.715481 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:14.715481 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:14.715481 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:33:14.715481 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:14.715481 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:33:14.715481 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:14.715481 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:14.715481 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:14.715481 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:14.715481 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:14.715481 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:14.715481 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:14.715481 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:14.715481 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:14.715481 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:14.715481 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:14.715481 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:14.715481 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:14.716437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:14.716437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:14.716437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:33:14.716437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:33:14.716437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:33:14.716437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:33:14.716437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:33:14.716437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:33:14.716437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 02:33:14.716437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:14.716437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 287/593] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:33:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:14.716437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac48197-19e2-41d7-b6c1-0b34cd525c45 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:14.824020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d1933-3293-471f-813b-ebd6fddaed0e None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:14.838554 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac48197-19e2-41d7-b6c1-0b34cd525c45 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:14.842634 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:14.875748 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba96a1a-7caf-40c0-81e1-2ea02871db7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:14.876780 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba96a1a-7caf-40c0-81e1-2ea02871db7a None admin] REQUEST_METHOD: `PUT` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:14.877383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba96a1a-7caf-40c0-81e1-2ea02871db7a None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:14.877666 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba96a1a-7caf-40c0-81e1-2ea02871db7a None admin] PATH_INFO: `/v3/projects/2a792f8dc45145d687ddd58c94d44b5e/users/6160520562b74be3b25c49a30e990d8b/roles/fb2fbee669e14ffbbe0aa70642eab1ce` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:14.888441 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b042221-66a2-4c5c-9b5e-90ba563d9d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:14.889548 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b042221-66a2-4c5c-9b5e-90ba563d9d3c None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:14.890028 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b042221-66a2-4c5c-9b5e-90ba563d9d3c None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:14.890604 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b042221-66a2-4c5c-9b5e-90ba563d9d3c None admin] PATH_INFO: `/v3/projects` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:14.891868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b042221-66a2-4c5c-9b5e-90ba563d9d3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:14.892424 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b042221-66a2-4c5c-9b5e-90ba563d9d3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:14.895559 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b042221-66a2-4c5c-9b5e-90ba563d9d3c None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:14.897139 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba96a1a-7caf-40c0-81e1-2ea02871db7a None admin] RBAC: Authorizing `identity:create_grant(project_id=2a792f8dc45145d687ddd58c94d44b5e, user_id=6160520562b74be3b25c49a30e990d8b, role_id=fb2fbee669e14ffbbe0aa70642eab1ce)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:14.899478 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba96a1a-7caf-40c0-81e1-2ea02871db7a None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:14.922941 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 307/594] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Sep 22 02:33:14 2022] PUT /identity/v3/projects/2a792f8dc45145d687ddd58c94d44b5e/users/6160520562b74be3b25c49a30e990d8b/roles/fb2fbee669e14ffbbe0aa70642eab1ce => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 02:33:14.922941 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba96a1a-7caf-40c0-81e1-2ea02871db7a None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:14.928426 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 288/595] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Sep 22 02:33:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:14.930922 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b042221-66a2-4c5c-9b5e-90ba563d9d3c None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:14.959724 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:15.111424 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b042221-66a2-4c5c-9b5e-90ba563d9d3c None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:15.127879 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba96a1a-7caf-40c0-81e1-2ea02871db7a None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:15.159020 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d018d808-19a8-4660-b7d3-24047fca8f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:15.159905 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018d808-19a8-4660-b7d3-24047fca8f42 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:15.160555 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018d808-19a8-4660-b7d3-24047fca8f42 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:15.160847 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018d808-19a8-4660-b7d3-24047fca8f42 None admin] PATH_INFO: `/v3/users` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:15.161615 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d018d808-19a8-4660-b7d3-24047fca8f42 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:15.162010 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d018d808-19a8-4660-b7d3-24047fca8f42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:15.163987 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d018d808-19a8-4660-b7d3-24047fca8f42 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:15.185809 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9eda6e-1080-4cfa-9481-59b6bf94ad6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:15.186522 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9eda6e-1080-4cfa-9481-59b6bf94ad6f None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:15.186756 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9eda6e-1080-4cfa-9481-59b6bf94ad6f None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:15.187073 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9eda6e-1080-4cfa-9481-59b6bf94ad6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:15.194846 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 289/596] 10.222.0.20 () {56 vars in 1137 bytes} [Thu Sep 22 02:33:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:15.196085 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d018d808-19a8-4660-b7d3-24047fca8f42 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:15.207486 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9eda6e-1080-4cfa-9481-59b6bf94ad6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:15.208969 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9eda6e-1080-4cfa-9481-59b6bf94ad6f None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:15.227824 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-6b9eda6e-1080-4cfa-9481-59b6bf94ad6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:15.227824 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:15.227824 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:15.227824 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:15.227824 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:15.227824 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:15.227824 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:15.227824 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:15.227824 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:15.227824 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:15.227824 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:15.227824 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:15.227824 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:15.227824 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:15.227824 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:15.227824 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:15.230272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:15.230272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:15.230272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:15.230272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:15.230272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:15.230272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:15.230272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:15.230272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:15.230272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:15.230272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:15.230272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:15.230272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:15.230272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:15.230272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:15.230272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:15.230272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:15.230272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:15.232518 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:15.232518 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:15.232518 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:15.232518 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 308/597] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:15.232518 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9eda6e-1080-4cfa-9481-59b6bf94ad6f None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:15.254998 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:15.325283 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:15.367283 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d018d808-19a8-4660-b7d3-24047fca8f42 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:15.405681 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9eda6e-1080-4cfa-9481-59b6bf94ad6f None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:15.407779 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b6954b-9260-424c-87d2-ddb434cc20a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:15.408770 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b6954b-9260-424c-87d2-ddb434cc20a5 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:15.409256 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b6954b-9260-424c-87d2-ddb434cc20a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:15.409727 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b6954b-9260-424c-87d2-ddb434cc20a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:15.410721 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b6954b-9260-424c-87d2-ddb434cc20a5 None admin] RBAC: Adding query filter params (role.id=fb2fbee669e14ffbbe0aa70642eab1ce, scope.project.id=2a792f8dc45145d687ddd58c94d44b5e, user.id=4b9ad548622b4e54a6fecd0afd5ef3c2) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:15.411379 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b6954b-9260-424c-87d2-ddb434cc20a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:15.413608 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b6954b-9260-424c-87d2-ddb434cc20a5 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:15.420744 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 290/598] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:33:15 2022] GET /identity/v3/role_assignments?user.id=4b9ad548622b4e54a6fecd0afd5ef3c2&scope.project.id=2a792f8dc45145d687ddd58c94d44b5e&role.id=fb2fbee669e14ffbbe0aa70642eab1ce => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:15.422043 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b6954b-9260-424c-87d2-ddb434cc20a5 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:15.476242 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8e0d4-f06d-439e-97d2-7ea6430531bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:15.476811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8e0d4-f06d-439e-97d2-7ea6430531bb None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:15.477137 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8e0d4-f06d-439e-97d2-7ea6430531bb None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:15.477422 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8e0d4-f06d-439e-97d2-7ea6430531bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:15.492996 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc8e0d4-f06d-439e-97d2-7ea6430531bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:15.495669 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc8e0d4-f06d-439e-97d2-7ea6430531bb None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:15.517315 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-5fc8e0d4-f06d-439e-97d2-7ea6430531bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:15.517315 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:15.517315 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:15.517315 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:15.517315 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:15.517315 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:15.517315 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:15.517315 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:15.517315 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:15.517315 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:15.517315 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:15.517315 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:15.517315 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:15.517315 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:15.517315 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:15.517315 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:15.518364 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:15.518364 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:15.518364 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:15.518364 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:15.518364 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:15.518364 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:15.518364 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:15.518364 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:15.518364 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:15.518364 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:15.518364 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:15.518364 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:15.518364 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:15.518364 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:15.518364 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:15.518364 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:15.518364 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:15.519307 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:15.519307 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:15.519307 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:15.526540 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 309/599] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:15.528324 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0a5ce-e3f7-430b-a98e-75cb4576867f None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:15.528848 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0a5ce-e3f7-430b-a98e-75cb4576867f None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:15.529278 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0a5ce-e3f7-430b-a98e-75cb4576867f None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:15.531589 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 310/600] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:33:15.532816 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe0a5ce-e3f7-430b-a98e-75cb4576867f None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:15.559743 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b6954b-9260-424c-87d2-ddb434cc20a5 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:15.608467 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae2fc4-83b2-41c2-9072-6c8dd65ab5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:15.609789 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae2fc4-83b2-41c2-9072-6c8dd65ab5c8 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:15.610386 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae2fc4-83b2-41c2-9072-6c8dd65ab5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:15.611143 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae2fc4-83b2-41c2-9072-6c8dd65ab5c8 None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:15.612501 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ae2fc4-83b2-41c2-9072-6c8dd65ab5c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:15.613252 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ae2fc4-83b2-41c2-9072-6c8dd65ab5c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:15.615604 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ae2fc4-83b2-41c2-9072-6c8dd65ab5c8 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:15.619741 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:15.634166 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 291/601] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:15.636231 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb8894-e8a2-43b9-9524-30d7fe1b4cdb None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:15.636851 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb8894-e8a2-43b9-9524-30d7fe1b4cdb None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:15.637608 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb8894-e8a2-43b9-9524-30d7fe1b4cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:15.641632 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:15.659781 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-9aeb8894-e8a2-43b9-9524-30d7fe1b4cdb None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:15.661429 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-9aeb8894-e8a2-43b9-9524-30d7fe1b4cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:15.687592 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe0a5ce-e3f7-430b-a98e-75cb4576867f None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:15.729513 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873df891-8753-4bba-8104-f2c5b4b728fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:15.730364 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873df891-8753-4bba-8104-f2c5b4b728fd None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:15.730859 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873df891-8753-4bba-8104-f2c5b4b728fd None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:15.731240 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873df891-8753-4bba-8104-f2c5b4b728fd None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:15.732061 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873df891-8753-4bba-8104-f2c5b4b728fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:15.732372 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873df891-8753-4bba-8104-f2c5b4b728fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:15.734507 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873df891-8753-4bba-8104-f2c5b4b728fd None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:15.754991 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 311/602] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:15.754991 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873df891-8753-4bba-8104-f2c5b4b728fd None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:15.779181 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 292/603] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:33:15.781338 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb8894-e8a2-43b9-9524-30d7fe1b4cdb None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:16.037109 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873df891-8753-4bba-8104-f2c5b4b728fd None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:16.054538 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb8894-e8a2-43b9-9524-30d7fe1b4cdb None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:16.097860 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:16.119981 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4760cf1-83db-4b50-8449-00ef50873f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:16.121168 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599ccaf-e349-4c98-b6b3-65cfd102783a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:16.121799 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599ccaf-e349-4c98-b6b3-65cfd102783a None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:16.122076 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599ccaf-e349-4c98-b6b3-65cfd102783a None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:16.122344 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599ccaf-e349-4c98-b6b3-65cfd102783a None admin] PATH_INFO: `/v3/projects/service` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:16.122978 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4760cf1-83db-4b50-8449-00ef50873f24 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:16.123495 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4760cf1-83db-4b50-8449-00ef50873f24 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:16.124148 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4760cf1-83db-4b50-8449-00ef50873f24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:16.127192 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:16.134123 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599ccaf-e349-4c98-b6b3-65cfd102783a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:16.135762 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599ccaf-e349-4c98-b6b3-65cfd102783a None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:16.137061 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4760cf1-83db-4b50-8449-00ef50873f24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:16.140003 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4760cf1-83db-4b50-8449-00ef50873f24 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:16.155936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-9599ccaf-e349-4c98-b6b3-65cfd102783a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:16.155936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:16.155936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:16.155936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:16.155936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:16.155936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:16.155936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:16.155936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:16.155936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:16.155936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:16.155936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:16.155936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:16.155936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:33:16.155936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:33:16.155936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:33:16.156922 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:33:16.156922 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:16.156922 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:16.156922 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:16.156922 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:16.156922 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:16.156922 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:16.156922 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:16.156922 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:16.156922 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:16.156922 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:16.156922 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:16.156922 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:16.156922 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:16.156922 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:16.156922 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:16.156922 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:16.157816 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:33:16.157816 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:33:16.157816 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:33:16.157816 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:33:16.157816 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:33:16.157816 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:33:16.157816 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:16.157816 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:16.157816 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 312/604] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:33:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:16.163716 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599ccaf-e349-4c98-b6b3-65cfd102783a None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:16.185908 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-c4760cf1-83db-4b50-8449-00ef50873f24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:16.185908 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:16.185908 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:16.185908 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:16.185908 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:16.185908 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:16.185908 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:16.185908 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:16.185908 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:16.185908 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:16.185908 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:16.185908 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:16.185908 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:33:16.185908 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:33:16.185908 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:33:16.186926 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:33:16.186926 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:16.186926 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:16.186926 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:16.186926 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:16.186926 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:16.186926 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:16.186926 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:16.186926 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:16.186926 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:16.186926 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:16.186926 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:16.186926 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:16.186926 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:16.186926 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:16.186926 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:16.186926 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:16.187823 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:33:16.187823 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:33:16.187823 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:33:16.187823 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:33:16.187823 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:33:16.187823 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:33:16.187823 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:16.187823 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:16.195021 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 293/605] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:33:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 408 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:16.196890 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4760cf1-83db-4b50-8449-00ef50873f24 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:16.275326 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599ccaf-e349-4c98-b6b3-65cfd102783a None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:16.313520 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c2826-e2ab-44a5-a972-0941391c538e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:16.314056 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c2826-e2ab-44a5-a972-0941391c538e None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:16.314335 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c2826-e2ab-44a5-a972-0941391c538e None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:16.314614 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c2826-e2ab-44a5-a972-0941391c538e None admin] PATH_INFO: `/v3/projects` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:16.315393 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c2826-e2ab-44a5-a972-0941391c538e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:16.315700 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c2826-e2ab-44a5-a972-0941391c538e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:16.317159 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c2826-e2ab-44a5-a972-0941391c538e None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:16.347197 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 313/606] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Sep 22 02:33:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:16.348636 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c2826-e2ab-44a5-a972-0941391c538e None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:16.372645 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4760cf1-83db-4b50-8449-00ef50873f24 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:16.411794 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f1fcc-4398-4f2c-9236-0c8f9414a8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:16.412615 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f1fcc-4398-4f2c-9236-0c8f9414a8e2 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:16.413545 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f1fcc-4398-4f2c-9236-0c8f9414a8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:16.413945 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f1fcc-4398-4f2c-9236-0c8f9414a8e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:16.426261 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f1fcc-4398-4f2c-9236-0c8f9414a8e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:33:16.432649 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-361f1fcc-4398-4f2c-9236-0c8f9414a8e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:33:16.432649 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:16.432649 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:16.432649 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:16.432649 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:16.432649 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:16.432649 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:16.432649 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:16.432649 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:16.432649 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:16.432649 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:16.432649 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:16.432649 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:33:16.432649 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:33:16.432649 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:33:16.432649 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:33:16.433866 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:33:16.433866 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:33:16.433866 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:33:16.433866 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:33:16.433866 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:16.433866 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:16.433866 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:16.433866 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:16.433866 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:16.433866 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:16.433866 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:16.433866 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:16.433866 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:16.433866 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:16.433866 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:16.433866 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:16.434897 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:16.434897 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:16.434897 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:16.434897 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:16.434897 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:33:16.434897 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:33:16.434897 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:33:16.434897 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:33:16.434897 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:33:16.434897 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:33:16.434897 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:33:16.434897 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:16.435633 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:16.436978 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 294/607] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Sep 22 02:33:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:16.443507 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f1fcc-4398-4f2c-9236-0c8f9414a8e2 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:16.500437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:16.508054 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c2826-e2ab-44a5-a972-0941391c538e None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:16.547714 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d68d4f-8943-4c15-9eaf-c668e8c096de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:16.548674 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d68d4f-8943-4c15-9eaf-c668e8c096de None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:16.549137 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d68d4f-8943-4c15-9eaf-c668e8c096de None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:16.549591 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d68d4f-8943-4c15-9eaf-c668e8c096de None admin] PATH_INFO: `/v3/projects` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:16.550502 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d68d4f-8943-4c15-9eaf-c668e8c096de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:16.551045 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d68d4f-8943-4c15-9eaf-c668e8c096de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:16.552754 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d68d4f-8943-4c15-9eaf-c668e8c096de None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:16.565484 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f1fcc-4398-4f2c-9236-0c8f9414a8e2 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:16.567042 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 314/608] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Sep 22 02:33:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:16.568126 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d68d4f-8943-4c15-9eaf-c668e8c096de None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:16.610460 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f187c3-026d-4876-ba1f-42105211824a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:16.611182 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f187c3-026d-4876-ba1f-42105211824a None admin] REQUEST_METHOD: `PUT` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:16.611468 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f187c3-026d-4876-ba1f-42105211824a None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:16.611748 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f187c3-026d-4876-ba1f-42105211824a None admin] PATH_INFO: `/v3/projects/2a792f8dc45145d687ddd58c94d44b5e/users/a0f635200ad241b3b31ed49701bc511b/roles/fb2fbee669e14ffbbe0aa70642eab1ce` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:16.630872 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f187c3-026d-4876-ba1f-42105211824a None admin] RBAC: Authorizing `identity:create_grant(project_id=2a792f8dc45145d687ddd58c94d44b5e, user_id=a0f635200ad241b3b31ed49701bc511b, role_id=fb2fbee669e14ffbbe0aa70642eab1ce)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:16.632750 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f187c3-026d-4876-ba1f-42105211824a None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:16.653114 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 295/609] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Sep 22 02:33:16 2022] PUT /identity/v3/projects/2a792f8dc45145d687ddd58c94d44b5e/users/a0f635200ad241b3b31ed49701bc511b/roles/fb2fbee669e14ffbbe0aa70642eab1ce => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 02:33:16.654321 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f187c3-026d-4876-ba1f-42105211824a None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:16.736267 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d68d4f-8943-4c15-9eaf-c668e8c096de None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:16.753788 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:16.804886 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9f9931-f1ce-4a72-99a2-4ef595cb964f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:16.805945 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f9931-f1ce-4a72-99a2-4ef595cb964f None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:16.806459 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f9931-f1ce-4a72-99a2-4ef595cb964f None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:16.811652 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f9931-f1ce-4a72-99a2-4ef595cb964f None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:16.813723 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9f9931-f1ce-4a72-99a2-4ef595cb964f None admin] RBAC: Adding query filter params (name=service) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:16.813723 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9f9931-f1ce-4a72-99a2-4ef595cb964f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:16.814065 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9f9931-f1ce-4a72-99a2-4ef595cb964f None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:16.846098 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 315/610] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Sep 22 02:33:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:16.852541 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9f9931-f1ce-4a72-99a2-4ef595cb964f None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:16.867537 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:16.876057 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f187c3-026d-4876-ba1f-42105211824a None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:16.922658 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79de1e-571e-48fd-a593-a7e3ac84d52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:16.923382 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79de1e-571e-48fd-a593-a7e3ac84d52d None admin] REQUEST_METHOD: `PUT` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:16.923670 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79de1e-571e-48fd-a593-a7e3ac84d52d None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:16.923944 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79de1e-571e-48fd-a593-a7e3ac84d52d None admin] PATH_INFO: `/v3/projects/2a792f8dc45145d687ddd58c94d44b5e/users/a08a0ed98bc14328a3d24309d7996b51/roles/fb2fbee669e14ffbbe0aa70642eab1ce` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:16.940686 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f79de1e-571e-48fd-a593-a7e3ac84d52d None admin] RBAC: Authorizing `identity:create_grant(project_id=2a792f8dc45145d687ddd58c94d44b5e, user_id=a08a0ed98bc14328a3d24309d7996b51, role_id=fb2fbee669e14ffbbe0aa70642eab1ce)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:16.942362 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f79de1e-571e-48fd-a593-a7e3ac84d52d None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:16.969815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 296/611] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Sep 22 02:33:16 2022] PUT /identity/v3/projects/2a792f8dc45145d687ddd58c94d44b5e/users/a08a0ed98bc14328a3d24309d7996b51/roles/fb2fbee669e14ffbbe0aa70642eab1ce => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 02:33:17.039012 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9f9931-f1ce-4a72-99a2-4ef595cb964f None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:17.106141 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed312d9-0c73-484f-900c-952856b722bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:17.107923 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed312d9-0c73-484f-900c-952856b722bf None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:17.108829 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed312d9-0c73-484f-900c-952856b722bf None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:17.109501 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed312d9-0c73-484f-900c-952856b722bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:17.139533 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed312d9-0c73-484f-900c-952856b722bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:17.142872 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed312d9-0c73-484f-900c-952856b722bf None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:17.166493 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-2ed312d9-0c73-484f-900c-952856b722bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:17.166493 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:17.166493 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:17.166493 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:17.166493 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:17.166493 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:17.166493 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:17.166493 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:17.166493 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:17.166493 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:17.166493 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:17.166493 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:17.166493 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:17.166493 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:17.166493 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:17.166493 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:17.172369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:17.172369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:17.172369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:17.172369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:17.172369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:17.172369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:17.172369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:17.172369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:17.172369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:17.172369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:17.172369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:17.172369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:17.172369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:17.172369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:17.172369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:17.172369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:17.172369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:17.174105 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:17.174105 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:17.174105 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:17.174105 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 316/612] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:33:17.184741 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79de1e-571e-48fd-a593-a7e3ac84d52d None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:17.395315 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79de1e-571e-48fd-a593-a7e3ac84d52d None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:17.477607 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d0d69-ae99-490a-ba3b-01a667627dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:17.478582 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d0d69-ae99-490a-ba3b-01a667627dd5 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:17.479121 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d0d69-ae99-490a-ba3b-01a667627dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:17.479594 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d0d69-ae99-490a-ba3b-01a667627dd5 None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:17.480842 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0d0d69-ae99-490a-ba3b-01a667627dd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:17.481400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0d0d69-ae99-490a-ba3b-01a667627dd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:17.484173 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0d0d69-ae99-490a-ba3b-01a667627dd5 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:17.515135 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 297/613] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:17.517466 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed312d9-0c73-484f-900c-952856b722bf None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:17.559713 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:17.729933 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed312d9-0c73-484f-900c-952856b722bf None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:17.812811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83619b42-aa9f-45f6-af01-129a56dec81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:17.813908 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83619b42-aa9f-45f6-af01-129a56dec81d None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:17.814398 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83619b42-aa9f-45f6-af01-129a56dec81d None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:17.814893 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83619b42-aa9f-45f6-af01-129a56dec81d None admin] PATH_INFO: `/v3/users/nova` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:17.827620 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83619b42-aa9f-45f6-af01-129a56dec81d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:17.829460 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83619b42-aa9f-45f6-af01-129a56dec81d None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:17.848118 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-83619b42-aa9f-45f6-af01-129a56dec81d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 02:33:17.848118 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:17.848118 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:17.848118 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:17.848118 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:17.848118 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:17.848118 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:17.848118 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:17.848118 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:17.848118 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:17.848118 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:17.848118 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:17.848118 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:33:17.848118 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:33:17.848118 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:33:17.848118 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:33:17.849180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:17.849180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:17.849180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:33:17.849180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:17.849180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:33:17.849180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:17.849180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:17.849180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:17.849180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:17.849180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:17.849180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:17.849180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:17.849180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:17.849180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:17.849180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:17.849180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:17.849180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:17.849180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:17.850108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:17.850108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:17.850108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:33:17.850108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:33:17.850108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:33:17.850108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:33:17.850108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:33:17.850108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:33:17.850108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 02:33:17.850108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:17.852803 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 317/614] 10.222.0.20 () {56 vars in 1093 bytes} [Thu Sep 22 02:33:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:17.863508 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d0d69-ae99-490a-ba3b-01a667627dd5 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:17.903334 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:17.987988 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb070e-1168-4b5a-8dfa-45581c7384e6 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:17.988166 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb070e-1168-4b5a-8dfa-45581c7384e6 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:17.988486 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb070e-1168-4b5a-8dfa-45581c7384e6 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:17.990117 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 318/615] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:33:17.996206 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd875e30-4691-47bf-a89f-27d6cbfb9d1e None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:17.996526 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd875e30-4691-47bf-a89f-27d6cbfb9d1e None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:17.996808 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd875e30-4691-47bf-a89f-27d6cbfb9d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:18.047964 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-cd875e30-4691-47bf-a89f-27d6cbfb9d1e None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:18.049215 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-cd875e30-4691-47bf-a89f-27d6cbfb9d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:18.093633 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d0d69-ae99-490a-ba3b-01a667627dd5 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:18.190085 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb18de-7baa-41c9-bd92-f49862924757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:18.193638 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb18de-7baa-41c9-bd92-f49862924757 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:18.194189 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb18de-7baa-41c9-bd92-f49862924757 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:18.195187 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb18de-7baa-41c9-bd92-f49862924757 None admin] PATH_INFO: `/v3/users` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:18.196602 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb18de-7baa-41c9-bd92-f49862924757 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:18.197153 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb18de-7baa-41c9-bd92-f49862924757 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:18.206224 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb18de-7baa-41c9-bd92-f49862924757 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:18.233487 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 319/616] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 02:33:18.263374 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 298/617] 10.222.0.20 () {56 vars in 1133 bytes} [Thu Sep 22 02:33:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:18.269728 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd875e30-4691-47bf-a89f-27d6cbfb9d1e None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:18.293437 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:18.377672 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d86c9-8af7-4cae-b82d-c1105469905d None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:18.378315 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d86c9-8af7-4cae-b82d-c1105469905d None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:18.378861 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d86c9-8af7-4cae-b82d-c1105469905d None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:18.380855 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 299/618] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:33:18.387702 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c3b181-41f4-4194-99ef-285e106d1356 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:18.388317 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c3b181-41f4-4194-99ef-285e106d1356 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:18.388789 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c3b181-41f4-4194-99ef-285e106d1356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:18.431534 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-d3c3b181-41f4-4194-99ef-285e106d1356 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:18.435193 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-d3c3b181-41f4-4194-99ef-285e106d1356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:18.521237 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd875e30-4691-47bf-a89f-27d6cbfb9d1e None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:18.542970 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:18.557627 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 300/619] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 02:33:18.592297 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63debb8c-86fa-469e-b50b-34a5f6b20c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:18.592930 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63debb8c-86fa-469e-b50b-34a5f6b20c47 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:18.593209 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63debb8c-86fa-469e-b50b-34a5f6b20c47 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:18.593473 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63debb8c-86fa-469e-b50b-34a5f6b20c47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:18.680040 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63debb8c-86fa-469e-b50b-34a5f6b20c47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:18.681515 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63debb8c-86fa-469e-b50b-34a5f6b20c47 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:18.695460 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:18.848106 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c3b181-41f4-4194-99ef-285e106d1356 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:18.906847 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 320/620] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:33:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 638 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 02:33:18.913012 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63debb8c-86fa-469e-b50b-34a5f6b20c47 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:19.095159 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c3b181-41f4-4194-99ef-285e106d1356 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:19.095550 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63debb8c-86fa-469e-b50b-34a5f6b20c47 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:19.109117 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:19.143742 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bc32bd-f45a-4f9b-adee-2ba56ceb05e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:19.144400 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc32bd-f45a-4f9b-adee-2ba56ceb05e3 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:19.144677 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc32bd-f45a-4f9b-adee-2ba56ceb05e3 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:19.144948 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc32bd-f45a-4f9b-adee-2ba56ceb05e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:19.158405 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bc32bd-f45a-4f9b-adee-2ba56ceb05e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:19.160064 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bc32bd-f45a-4f9b-adee-2ba56ceb05e3 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:19.168066 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea19d0d-4ebc-4bf6-bf37-e417708b73be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:19.169473 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea19d0d-4ebc-4bf6-bf37-e417708b73be None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:19.170084 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea19d0d-4ebc-4bf6-bf37-e417708b73be None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:19.170917 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea19d0d-4ebc-4bf6-bf37-e417708b73be None admin] PATH_INFO: `/v3/roles/service` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:19.178294 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-c0bc32bd-f45a-4f9b-adee-2ba56ceb05e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:19.178294 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:19.178294 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:19.178294 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:19.178294 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:19.178294 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:19.178294 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:19.178294 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:19.178294 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:19.178294 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:19.178294 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:19.178294 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:19.178294 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:19.178294 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:19.178294 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:19.178294 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:19.179903 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:19.179903 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:19.179903 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:19.179903 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:19.179903 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:19.179903 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:19.179903 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:19.179903 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:19.179903 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:19.179903 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:19.179903 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:19.179903 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:19.179903 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:19.179903 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:19.179903 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:19.179903 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:19.179903 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:19.180806 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:19.180806 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:19.180806 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:19.180806 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 321/621] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:33:19.181044 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bc32bd-f45a-4f9b-adee-2ba56ceb05e3 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:19.200001 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea19d0d-4ebc-4bf6-bf37-e417708b73be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:33:19.210868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-2ea19d0d-4ebc-4bf6-bf37-e417708b73be None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:33:19.210868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:19.210868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:19.210868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:19.210868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:19.210868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:19.210868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:19.210868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:19.210868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:19.210868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:19.210868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:19.210868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:19.210868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:33:19.210868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:33:19.210868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:33:19.210868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:33:19.212868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:33:19.212868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:33:19.212868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:33:19.212868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:33:19.212868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:19.212868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:19.212868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:19.212868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:19.212868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:19.212868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:19.212868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:19.212868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:19.212868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:19.212868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:19.212868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:19.212868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:19.214645 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:19.214645 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:19.214645 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:19.214645 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:19.214645 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:33:19.214645 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:33:19.214645 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:33:19.214645 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:33:19.214645 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:33:19.214645 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:33:19.214645 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:33:19.214645 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:19.222853 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 301/622] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Sep 22 02:33:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:33:19.222853 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea19d0d-4ebc-4bf6-bf37-e417708b73be None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:19.300206 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:19.314980 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bc32bd-f45a-4f9b-adee-2ba56ceb05e3 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:19.354210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a51f4db-da01-4053-a7c4-5b433f70b2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:19.355411 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51f4db-da01-4053-a7c4-5b433f70b2f4 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:19.355877 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51f4db-da01-4053-a7c4-5b433f70b2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:19.356329 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51f4db-da01-4053-a7c4-5b433f70b2f4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:19.374071 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a51f4db-da01-4053-a7c4-5b433f70b2f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:33:19.385586 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea19d0d-4ebc-4bf6-bf37-e417708b73be None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:19.385927 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-2a51f4db-da01-4053-a7c4-5b433f70b2f4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:33:19.385927 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:19.385927 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:19.385927 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:19.385927 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:19.385927 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:19.385927 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:19.385927 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:19.385927 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:19.385927 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:19.385927 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:19.385927 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:19.385927 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:33:19.385927 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:33:19.385927 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:33:19.385927 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:33:19.386866 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:33:19.386866 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:33:19.386866 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:33:19.386866 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:33:19.386866 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:19.386866 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:19.386866 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:19.386866 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:19.386866 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:19.386866 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:19.386866 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:19.386866 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:19.386866 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:19.386866 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:19.386866 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:19.386866 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:19.387722 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:19.387722 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:19.387722 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:19.387722 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:19.387722 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:33:19.387722 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:33:19.387722 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:33:19.387722 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:33:19.387722 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:33:19.387722 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:33:19.387722 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:33:19.387722 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:19.388371 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 322/623] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Sep 22 02:33:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:19.389151 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a51f4db-da01-4053-a7c4-5b433f70b2f4 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:19.433605 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5fe090-bc3f-4a1f-89ca-c69ea29a0d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:19.434629 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fe090-bc3f-4a1f-89ca-c69ea29a0d08 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:19.435083 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fe090-bc3f-4a1f-89ca-c69ea29a0d08 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:19.435551 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fe090-bc3f-4a1f-89ca-c69ea29a0d08 None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:19.435715 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:19.436671 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5fe090-bc3f-4a1f-89ca-c69ea29a0d08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:19.436998 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5fe090-bc3f-4a1f-89ca-c69ea29a0d08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:19.439084 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5fe090-bc3f-4a1f-89ca-c69ea29a0d08 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:19.454781 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 302/624] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:19.457125 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd7dee-04f6-412e-9ef9-5d9d6d9b2848 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:19.457626 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd7dee-04f6-412e-9ef9-5d9d6d9b2848 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:19.458106 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd7dee-04f6-412e-9ef9-5d9d6d9b2848 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:19.466418 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 303/625] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:19 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:33:19.467512 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd7dee-04f6-412e-9ef9-5d9d6d9b2848 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:19.593056 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a51f4db-da01-4053-a7c4-5b433f70b2f4 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:19.610347 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd7dee-04f6-412e-9ef9-5d9d6d9b2848 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:19.631872 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:19.650158 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde488e5-1e01-43ed-a6f3-52ca6c4dc81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:19.650793 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde488e5-1e01-43ed-a6f3-52ca6c4dc81f None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:19.651133 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde488e5-1e01-43ed-a6f3-52ca6c4dc81f None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:19.651410 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde488e5-1e01-43ed-a6f3-52ca6c4dc81f None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:19.652129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde488e5-1e01-43ed-a6f3-52ca6c4dc81f None admin] RBAC: Adding query filter params (name=service) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:19.652644 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde488e5-1e01-43ed-a6f3-52ca6c4dc81f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:19.653836 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde488e5-1e01-43ed-a6f3-52ca6c4dc81f None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:19.671542 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 323/626] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Sep 22 02:33:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:19.671542 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde488e5-1e01-43ed-a6f3-52ca6c4dc81f None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:19.682885 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5bdbf-946a-47e1-ab03-4498b39d19ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:19.684011 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5bdbf-946a-47e1-ab03-4498b39d19ae None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:19.684544 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5bdbf-946a-47e1-ab03-4498b39d19ae None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:19.685078 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5bdbf-946a-47e1-ab03-4498b39d19ae None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:19.686338 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d5bdbf-946a-47e1-ab03-4498b39d19ae None admin] RBAC: Adding query filter params (name=service) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:19.686988 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d5bdbf-946a-47e1-ab03-4498b39d19ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:19.689059 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d5bdbf-946a-47e1-ab03-4498b39d19ae None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:19.704645 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 304/627] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Sep 22 02:33:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:19.706911 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e01bd-c7ea-474a-928d-ef0d190c9003 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:19.707505 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e01bd-c7ea-474a-928d-ef0d190c9003 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:19.708080 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e01bd-c7ea-474a-928d-ef0d190c9003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:19.742414 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-0e3e01bd-c7ea-474a-928d-ef0d190c9003 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:19.744225 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-0e3e01bd-c7ea-474a-928d-ef0d190c9003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:19.833697 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:19.860147 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 305/628] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:33:19.861045 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e01bd-c7ea-474a-928d-ef0d190c9003 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:19.863689 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde488e5-1e01-43ed-a6f3-52ca6c4dc81f None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:19.906126 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8490ab-4bf1-447d-b50e-118cfc54579c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:19.906667 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8490ab-4bf1-447d-b50e-118cfc54579c None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:19.906994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8490ab-4bf1-447d-b50e-118cfc54579c None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:19.907267 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8490ab-4bf1-447d-b50e-118cfc54579c None admin] PATH_INFO: `/v3/projects/service` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:19.915090 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8490ab-4bf1-447d-b50e-118cfc54579c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:19.916556 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8490ab-4bf1-447d-b50e-118cfc54579c None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:19.929758 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-4d8490ab-4bf1-447d-b50e-118cfc54579c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:19.929758 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:19.929758 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:19.929758 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:19.929758 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:19.929758 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:19.929758 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:19.929758 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:19.929758 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:19.929758 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:19.929758 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:19.929758 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:19.929758 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:33:19.929758 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:33:19.929758 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:33:19.931383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:33:19.931383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:19.931383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:19.931383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:19.931383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:19.931383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:19.931383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:19.931383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:19.931383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:19.931383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:19.931383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:19.931383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:19.931383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:19.931383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:19.931383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:19.931383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:19.931383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:19.932872 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:33:19.932872 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:33:19.932872 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:33:19.932872 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:33:19.932872 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:33:19.932872 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:33:19.932872 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:19.932872 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:19.935398 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 324/629] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:33:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:19.937784 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8490ab-4bf1-447d-b50e-118cfc54579c None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:20.023478 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e01bd-c7ea-474a-928d-ef0d190c9003 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:20.033613 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:20.075848 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0447852d-bc04-4d8a-ab0f-ab7bdce832c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:20.076512 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447852d-bc04-4d8a-ab0f-ab7bdce832c5 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:20.076821 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447852d-bc04-4d8a-ab0f-ab7bdce832c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:20.077102 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447852d-bc04-4d8a-ab0f-ab7bdce832c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:20.089161 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0447852d-bc04-4d8a-ab0f-ab7bdce832c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:20.090648 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0447852d-bc04-4d8a-ab0f-ab7bdce832c5 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:20.107844 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-0447852d-bc04-4d8a-ab0f-ab7bdce832c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:20.107844 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:20.107844 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:20.107844 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:20.107844 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:20.107844 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:20.107844 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:20.107844 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:20.107844 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:20.107844 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:20.107844 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:20.107844 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:20.107844 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:20.107844 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:20.107844 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:20.107844 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:20.109722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:20.109722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:20.109722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:20.109722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:20.109722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:20.109722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:20.109722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:20.109722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:20.109722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:20.109722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:20.109722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:20.109722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:20.109722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:20.109722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:20.109722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:20.109722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:20.109722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:20.112911 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:20.112911 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:20.112911 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:20.114103 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 306/630] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:20.115608 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef3a23-a959-422d-b817-fb2b3e1048f5 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:20.115933 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef3a23-a959-422d-b817-fb2b3e1048f5 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:20.116190 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef3a23-a959-422d-b817-fb2b3e1048f5 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:20.117993 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 307/631] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:33:20.118436 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faef3a23-a959-422d-b817-fb2b3e1048f5 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:20.141000 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8490ab-4bf1-447d-b50e-118cfc54579c None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:20.195734 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193d772-b1d0-4042-a61a-e89cfc352780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:20.196478 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193d772-b1d0-4042-a61a-e89cfc352780 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:20.196762 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193d772-b1d0-4042-a61a-e89cfc352780 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:20.197037 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193d772-b1d0-4042-a61a-e89cfc352780 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:20.208472 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f193d772-b1d0-4042-a61a-e89cfc352780 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:20.210653 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f193d772-b1d0-4042-a61a-e89cfc352780 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:20.213639 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:20.228180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-f193d772-b1d0-4042-a61a-e89cfc352780 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:20.228180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:20.228180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:20.228180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:20.228180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:20.228180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:20.228180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:20.228180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:20.228180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:20.228180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:20.228180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:20.228180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:20.228180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:20.228180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:20.228180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:20.228180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:20.230261 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:20.230261 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:20.230261 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:20.230261 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:20.230261 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:20.230261 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:20.230261 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:20.230261 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:20.230261 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:20.230261 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:20.230261 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:20.230261 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:20.230261 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:20.230261 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:20.230261 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:20.230261 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:20.230261 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:20.232329 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:20.232329 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:20.232329 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:20.232329 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 325/632] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:20.232329 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193d772-b1d0-4042-a61a-e89cfc352780 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:20.294780 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faef3a23-a959-422d-b817-fb2b3e1048f5 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:20.339812 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d31cb-375e-4fe5-a2cc-7701d8e82aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:20.340793 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d31cb-375e-4fe5-a2cc-7701d8e82aa1 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:20.341265 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d31cb-375e-4fe5-a2cc-7701d8e82aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:20.341720 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d31cb-375e-4fe5-a2cc-7701d8e82aa1 None admin] PATH_INFO: `/v3/projects` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:20.342962 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d31cb-375e-4fe5-a2cc-7701d8e82aa1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:20.343486 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d31cb-375e-4fe5-a2cc-7701d8e82aa1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:20.346020 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d31cb-375e-4fe5-a2cc-7701d8e82aa1 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:20.348919 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193d772-b1d0-4042-a61a-e89cfc352780 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:20.365428 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 308/633] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Sep 22 02:33:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:20.368228 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db22915-c61a-4d2d-b751-ce5e1897d15d None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:20.368732 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db22915-c61a-4d2d-b751-ce5e1897d15d None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:20.369316 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db22915-c61a-4d2d-b751-ce5e1897d15d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:20.390026 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-2db22915-c61a-4d2d-b751-ce5e1897d15d None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:20.391316 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-2db22915-c61a-4d2d-b751-ce5e1897d15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:20.393666 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9052ec16-1e0e-4a26-af3f-1eab5789bf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:20.394333 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9052ec16-1e0e-4a26-af3f-1eab5789bf81 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:20.394614 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9052ec16-1e0e-4a26-af3f-1eab5789bf81 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:20.394925 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9052ec16-1e0e-4a26-af3f-1eab5789bf81 None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:20.395587 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9052ec16-1e0e-4a26-af3f-1eab5789bf81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:20.395892 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9052ec16-1e0e-4a26-af3f-1eab5789bf81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:20.397816 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9052ec16-1e0e-4a26-af3f-1eab5789bf81 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:20.407599 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:20.412215 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 326/634] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:20.413684 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9052ec16-1e0e-4a26-af3f-1eab5789bf81 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:20.500043 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:20.500680 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 309/635] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:33:20.501948 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db22915-c61a-4d2d-b751-ce5e1897d15d None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:20.577082 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9052ec16-1e0e-4a26-af3f-1eab5789bf81 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:20.655253 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e261f063-e6eb-47e2-8487-09fbcbab228b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:20.656481 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261f063-e6eb-47e2-8487-09fbcbab228b None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:20.656819 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261f063-e6eb-47e2-8487-09fbcbab228b None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:20.657311 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261f063-e6eb-47e2-8487-09fbcbab228b None admin] PATH_INFO: `/v3/roles/service` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:20.676713 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db22915-c61a-4d2d-b751-ce5e1897d15d None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:20.677043 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e261f063-e6eb-47e2-8487-09fbcbab228b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:33:20.684544 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-e261f063-e6eb-47e2-8487-09fbcbab228b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:33:20.684544 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:20.684544 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:20.684544 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:20.684544 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:20.684544 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:20.684544 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:20.684544 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:20.684544 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:20.684544 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:20.684544 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:20.684544 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:20.684544 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:33:20.684544 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:33:20.684544 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:33:20.684544 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:33:20.685570 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:33:20.685570 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:33:20.685570 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:33:20.685570 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:33:20.685570 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:20.685570 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:20.685570 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:20.685570 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:20.685570 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:20.685570 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:20.685570 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:20.685570 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:20.685570 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:20.685570 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:20.685570 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:20.685570 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:20.686405 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:20.686405 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:20.686405 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:20.686405 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:20.686405 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:33:20.686405 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:33:20.686405 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:33:20.686405 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:33:20.686405 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:33:20.686405 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:33:20.686405 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:33:20.686405 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:20.686405 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 327/636] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Sep 22 02:33:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:20.687773 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e261f063-e6eb-47e2-8487-09fbcbab228b None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:20.734672 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802cdf5-bd21-448a-a0fe-73ba05d5a2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:20.735894 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802cdf5-bd21-448a-a0fe-73ba05d5a2e6 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:20.736335 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802cdf5-bd21-448a-a0fe-73ba05d5a2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:20.737193 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802cdf5-bd21-448a-a0fe-73ba05d5a2e6 None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:20.738416 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c802cdf5-bd21-448a-a0fe-73ba05d5a2e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:20.738963 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c802cdf5-bd21-448a-a0fe-73ba05d5a2e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:20.740798 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c802cdf5-bd21-448a-a0fe-73ba05d5a2e6 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:20.757985 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 310/637] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:20.758618 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802cdf5-bd21-448a-a0fe-73ba05d5a2e6 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:20.771020 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:20.875328 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e261f063-e6eb-47e2-8487-09fbcbab228b None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:20.899597 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:20.925358 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3858ac-4f4c-4d3e-a017-2410c74f52f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:20.926111 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3858ac-4f4c-4d3e-a017-2410c74f52f9 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:20.926390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802cdf5-bd21-448a-a0fe-73ba05d5a2e6 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:20.926602 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3858ac-4f4c-4d3e-a017-2410c74f52f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:20.926978 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3858ac-4f4c-4d3e-a017-2410c74f52f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:20.927823 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3858ac-4f4c-4d3e-a017-2410c74f52f9 None admin] RBAC: Adding query filter params (role.id=fb2fbee669e14ffbbe0aa70642eab1ce, scope.project.id=2a792f8dc45145d687ddd58c94d44b5e, user.id=a64ed6c8e15d479887ab1dc59997d715) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:20.928239 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3858ac-4f4c-4d3e-a017-2410c74f52f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:20.930373 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3858ac-4f4c-4d3e-a017-2410c74f52f9 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:20.937305 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 328/638] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:33:20 2022] GET /identity/v3/role_assignments?user.id=a64ed6c8e15d479887ab1dc59997d715&scope.project.id=2a792f8dc45145d687ddd58c94d44b5e&role.id=fb2fbee669e14ffbbe0aa70642eab1ce => generated 608 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:20.938369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3858ac-4f4c-4d3e-a017-2410c74f52f9 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:20.990317 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56e158-320e-4633-ad58-eb8b74e1b766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:20.991784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56e158-320e-4633-ad58-eb8b74e1b766 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:20.992207 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56e158-320e-4633-ad58-eb8b74e1b766 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:20.992618 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56e158-320e-4633-ad58-eb8b74e1b766 None admin] PATH_INFO: `/v3/users/glance` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:21.007474 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c56e158-320e-4633-ad58-eb8b74e1b766 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:21.010087 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c56e158-320e-4633-ad58-eb8b74e1b766 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:21.036092 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-2c56e158-320e-4633-ad58-eb8b74e1b766 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 22 02:33:21.036092 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:21.036092 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:21.036092 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:21.036092 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:21.036092 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:21.036092 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:21.036092 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:21.036092 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:21.036092 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:21.036092 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:21.036092 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:21.036092 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:33:21.036092 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:33:21.036092 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:33:21.036092 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:33:21.037401 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:21.037401 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:21.037401 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:33:21.037401 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:21.037401 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:33:21.037401 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:21.037401 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:21.037401 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:21.037401 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:21.037401 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:21.037401 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:21.037401 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:21.037401 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:21.037401 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:21.037401 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:21.037401 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:21.037401 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:21.037401 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:21.040119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:21.040119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:21.040119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:33:21.040119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:33:21.040119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:33:21.040119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:33:21.040119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:33:21.040119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:33:21.040119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 22 02:33:21.040119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:21.040119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 311/639] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:33:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:21.040119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56e158-320e-4633-ad58-eb8b74e1b766 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:21.108608 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3858ac-4f4c-4d3e-a017-2410c74f52f9 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:21.159323 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:21.175858 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000b7a7-cb33-4be4-83fa-f3d5d569f6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:21.176506 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000b7a7-cb33-4be4-83fa-f3d5d569f6a7 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:21.177013 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000b7a7-cb33-4be4-83fa-f3d5d569f6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:21.177498 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000b7a7-cb33-4be4-83fa-f3d5d569f6a7 None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:21.179307 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5000b7a7-cb33-4be4-83fa-f3d5d569f6a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:21.180302 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5000b7a7-cb33-4be4-83fa-f3d5d569f6a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:21.182544 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5000b7a7-cb33-4be4-83fa-f3d5d569f6a7 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:21.197068 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 329/640] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Sep 22 02:33:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:21.198075 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000b7a7-cb33-4be4-83fa-f3d5d569f6a7 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:21.214318 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56e158-320e-4633-ad58-eb8b74e1b766 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:21.268306 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53eb46b-3cca-424f-b52c-9e4d65a2c51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:21.269253 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53eb46b-3cca-424f-b52c-9e4d65a2c51e None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:21.269763 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53eb46b-3cca-424f-b52c-9e4d65a2c51e None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:21.270258 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53eb46b-3cca-424f-b52c-9e4d65a2c51e None admin] PATH_INFO: `/v3/users/swift` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:21.277231 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:21.285161 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53eb46b-3cca-424f-b52c-9e4d65a2c51e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:21.287987 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53eb46b-3cca-424f-b52c-9e4d65a2c51e None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:21.310569 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-b53eb46b-3cca-424f-b52c-9e4d65a2c51e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 22 02:33:21.310569 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:21.310569 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:21.310569 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:21.310569 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:21.310569 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:21.310569 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:21.310569 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:21.310569 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:21.310569 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:21.310569 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:21.310569 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:21.310569 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:33:21.310569 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:33:21.310569 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:33:21.310569 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:33:21.312591 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:21.312591 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:21.312591 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:33:21.312591 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:21.312591 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:33:21.312591 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:21.312591 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:21.312591 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:21.312591 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:21.312591 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:21.312591 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:21.312591 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:21.312591 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:21.312591 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:21.312591 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:21.312591 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:21.312591 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:21.312591 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:21.315412 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:21.315412 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:21.315412 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:33:21.315412 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:33:21.315412 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:33:21.315412 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:33:21.315412 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:33:21.315412 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:33:21.315412 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 22 02:33:21.315412 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:21.315412 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 312/641] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Sep 22 02:33:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:21.315412 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53eb46b-3cca-424f-b52c-9e4d65a2c51e None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:21.358965 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000b7a7-cb33-4be4-83fa-f3d5d569f6a7 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:21.405047 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b36dcc0-13bf-4a3a-8c89-74b9361c7d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:21.406339 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36dcc0-13bf-4a3a-8c89-74b9361c7d61 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:21.406990 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36dcc0-13bf-4a3a-8c89-74b9361c7d61 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:21.407553 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36dcc0-13bf-4a3a-8c89-74b9361c7d61 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:21.428106 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b36dcc0-13bf-4a3a-8c89-74b9361c7d61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:33:21.439210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-7b36dcc0-13bf-4a3a-8c89-74b9361c7d61 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:33:21.439210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:21.439210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:21.439210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:21.439210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:21.439210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:21.439210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:21.439210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:21.439210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:21.439210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:21.439210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:21.439210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:21.439210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:33:21.439210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:33:21.439210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:33:21.439210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:33:21.440464 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:33:21.440464 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:33:21.440464 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:33:21.440464 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:33:21.440464 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:21.440464 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:21.440464 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:21.440464 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:21.440464 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:21.440464 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:21.440464 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:21.440464 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:21.440464 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:21.440464 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:21.440464 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:21.440464 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:21.441442 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:21.441442 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:21.441442 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:21.441442 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:21.441442 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:33:21.441442 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:33:21.441442 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:33:21.441442 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:33:21.441442 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:33:21.441442 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:33:21.441442 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:33:21.441442 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:21.442124 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 330/642] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Sep 22 02:33:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:21.443067 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b36dcc0-13bf-4a3a-8c89-74b9361c7d61 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:21.492146 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53eb46b-3cca-424f-b52c-9e4d65a2c51e None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:21.524418 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:21.535065 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdfb475-f3ad-4cf7-86e8-d74b4ceaa17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:21.535816 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdfb475-f3ad-4cf7-86e8-d74b4ceaa17f None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:21.536093 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdfb475-f3ad-4cf7-86e8-d74b4ceaa17f None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:21.536373 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdfb475-f3ad-4cf7-86e8-d74b4ceaa17f None admin] PATH_INFO: `/v3/users` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:21.537125 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdfb475-f3ad-4cf7-86e8-d74b4ceaa17f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:21.537436 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdfb475-f3ad-4cf7-86e8-d74b4ceaa17f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:21.539490 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdfb475-f3ad-4cf7-86e8-d74b4ceaa17f None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:21.574870 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 313/643] 10.222.0.20 () {56 vars in 1137 bytes} [Thu Sep 22 02:33:21 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:21.574870 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdfb475-f3ad-4cf7-86e8-d74b4ceaa17f None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:21.644370 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b36dcc0-13bf-4a3a-8c89-74b9361c7d61 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:21.647256 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:21.711378 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4783260-c630-4119-84cd-ca0b612372aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:21.712445 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4783260-c630-4119-84cd-ca0b612372aa None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:21.712942 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4783260-c630-4119-84cd-ca0b612372aa None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:21.713424 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4783260-c630-4119-84cd-ca0b612372aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:21.727445 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdfb475-f3ad-4cf7-86e8-d74b4ceaa17f None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:21.735025 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4783260-c630-4119-84cd-ca0b612372aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:21.738226 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4783260-c630-4119-84cd-ca0b612372aa None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:21.767888 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-c4783260-c630-4119-84cd-ca0b612372aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:21.767888 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:21.767888 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:21.767888 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:21.767888 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:21.767888 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:21.767888 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:21.767888 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:21.767888 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:21.767888 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:21.767888 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:21.767888 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:21.767888 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:21.767888 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:21.767888 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:21.767888 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:21.769681 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:21.769681 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:21.769681 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:21.769681 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:21.769681 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:21.769681 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:21.769681 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:21.769681 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:21.769681 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:21.769681 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:21.769681 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:21.769681 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:21.769681 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:21.769681 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:21.769681 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:21.769681 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:21.769681 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:21.771402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:21.771402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:21.771402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:21.779688 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 331/644] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:21.779688 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4783260-c630-4119-84cd-ca0b612372aa None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:21.782645 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037ba3a8-42a3-4a65-8459-9ade7b223a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:21.783663 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ba3a8-42a3-4a65-8459-9ade7b223a59 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:21.784123 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ba3a8-42a3-4a65-8459-9ade7b223a59 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:21.784749 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ba3a8-42a3-4a65-8459-9ade7b223a59 None admin] PATH_INFO: `/v3/users` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:21.785849 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037ba3a8-42a3-4a65-8459-9ade7b223a59 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:21.786379 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037ba3a8-42a3-4a65-8459-9ade7b223a59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:21.789580 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037ba3a8-42a3-4a65-8459-9ade7b223a59 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:21.813043 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 314/645] 10.222.0.20 () {56 vars in 1135 bytes} [Thu Sep 22 02:33:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:21.814494 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037ba3a8-42a3-4a65-8459-9ade7b223a59 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:21.878040 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:21.947721 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037ba3a8-42a3-4a65-8459-9ade7b223a59 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:21.987017 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108bf3e1-672e-483e-a7f6-40ca298ae53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:21.987654 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108bf3e1-672e-483e-a7f6-40ca298ae53d None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:21.987943 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108bf3e1-672e-483e-a7f6-40ca298ae53d None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:21.988225 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108bf3e1-672e-483e-a7f6-40ca298ae53d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:21.988672 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4783260-c630-4119-84cd-ca0b612372aa None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:22.061269 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad4db0-479e-446f-beda-96c14555a8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:22.062211 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad4db0-479e-446f-beda-96c14555a8dc None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:22.062597 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad4db0-479e-446f-beda-96c14555a8dc None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:22.063019 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad4db0-479e-446f-beda-96c14555a8dc None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:22.063889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ad4db0-479e-446f-beda-96c14555a8dc None admin] RBAC: Adding query filter params (name=service) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:22.064380 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ad4db0-479e-446f-beda-96c14555a8dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:22.066020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ad4db0-479e-446f-beda-96c14555a8dc None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:22.077880 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 332/646] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Sep 22 02:33:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:22.079475 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad4db0-479e-446f-beda-96c14555a8dc None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:22.103942 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108bf3e1-672e-483e-a7f6-40ca298ae53d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:22.105595 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108bf3e1-672e-483e-a7f6-40ca298ae53d None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:22.120343 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:22.165825 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:22.261184 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad4db0-479e-446f-beda-96c14555a8dc None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:22.265508 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 315/647] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:33:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 02:33:22.267028 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108bf3e1-672e-483e-a7f6-40ca298ae53d None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:22.310556 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b26a02-d41a-48f5-9351-7033d4a2ec76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:22.311841 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b26a02-d41a-48f5-9351-7033d4a2ec76 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:22.312259 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b26a02-d41a-48f5-9351-7033d4a2ec76 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:22.312678 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b26a02-d41a-48f5-9351-7033d4a2ec76 None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:22.313717 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b26a02-d41a-48f5-9351-7033d4a2ec76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:22.314178 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b26a02-d41a-48f5-9351-7033d4a2ec76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:22.316099 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b26a02-d41a-48f5-9351-7033d4a2ec76 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:22.337378 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 333/648] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:22.337378 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b26a02-d41a-48f5-9351-7033d4a2ec76 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:22.439573 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:22.463547 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108bf3e1-672e-483e-a7f6-40ca298ae53d None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:22.473729 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b26a02-d41a-48f5-9351-7033d4a2ec76 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:22.530698 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a728eeb-4865-498b-8bd4-388458efc216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:22.531848 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a728eeb-4865-498b-8bd4-388458efc216 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:22.532155 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a728eeb-4865-498b-8bd4-388458efc216 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:22.532659 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a728eeb-4865-498b-8bd4-388458efc216 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:22.538145 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ef375-0ee5-48f0-9980-570d5581fc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:22.539285 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ef375-0ee5-48f0-9980-570d5581fc4c None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:22.539779 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ef375-0ee5-48f0-9980-570d5581fc4c None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:22.540259 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ef375-0ee5-48f0-9980-570d5581fc4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:22.546701 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a728eeb-4865-498b-8bd4-388458efc216 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:22.548222 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a728eeb-4865-498b-8bd4-388458efc216 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:22.554975 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:22.555872 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ef375-0ee5-48f0-9980-570d5581fc4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:22.557550 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ef375-0ee5-48f0-9980-570d5581fc4c None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:22.579077 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-748ef375-0ee5-48f0-9980-570d5581fc4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:22.579077 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:22.579077 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:22.579077 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:22.579077 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:22.579077 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:22.579077 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:22.579077 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:22.579077 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:22.579077 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:22.579077 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:22.579077 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:22.579077 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:22.579077 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:22.579077 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:22.579077 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:22.581007 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:22.581007 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:22.581007 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:22.581007 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:22.581007 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:22.581007 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:22.581007 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:22.581007 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:22.581007 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:22.581007 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:22.581007 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:22.581007 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:22.581007 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:22.581007 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:22.581007 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:22.581007 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:22.581007 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:22.588998 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:22.588998 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:22.588998 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:22.589293 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-5a728eeb-4865-498b-8bd4-388458efc216 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:22.589293 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:22.589293 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:22.589293 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:22.589293 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:22.589293 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:22.589293 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:22.589293 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:22.589293 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:22.589293 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:22.589293 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:22.589293 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:22.589293 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:22.589293 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:22.589293 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:22.589293 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:22.591390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:22.591390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:22.591390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:22.591390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:22.591390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:22.591390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:22.591390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:22.591390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:22.591390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:22.591390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:22.591390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:22.591390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:22.591390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:22.591390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:22.591390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:22.591390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:22.591390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:22.595252 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:22.595252 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:22.595252 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:22.596815 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 334/649] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:22.596815 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ef375-0ee5-48f0-9980-570d5581fc4c None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:22.597545 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 316/650] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:22.597545 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a728eeb-4865-498b-8bd4-388458efc216 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:22.754419 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a728eeb-4865-498b-8bd4-388458efc216 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:22.760648 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ef375-0ee5-48f0-9980-570d5581fc4c None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:22.796015 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:22.808490 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7c2e0-e6f5-4660-8a95-0320b1985259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:22.809351 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7c2e0-e6f5-4660-8a95-0320b1985259 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:22.809821 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7c2e0-e6f5-4660-8a95-0320b1985259 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:22.810322 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7c2e0-e6f5-4660-8a95-0320b1985259 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:22.814778 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098d6587-3899-4ebf-8e60-78cd11a960bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:22.815899 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d6587-3899-4ebf-8e60-78cd11a960bc None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:22.816383 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d6587-3899-4ebf-8e60-78cd11a960bc None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:22.816858 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d6587-3899-4ebf-8e60-78cd11a960bc None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:22.825054 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed7c2e0-e6f5-4660-8a95-0320b1985259 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:22.826732 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed7c2e0-e6f5-4660-8a95-0320b1985259 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:22.831289 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098d6587-3899-4ebf-8e60-78cd11a960bc None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:22.833884 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098d6587-3899-4ebf-8e60-78cd11a960bc None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:22.844468 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-4ed7c2e0-e6f5-4660-8a95-0320b1985259 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:22.844468 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:22.844468 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:22.844468 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:22.844468 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:22.844468 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:22.844468 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:22.844468 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:22.844468 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:22.844468 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:22.844468 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:22.844468 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:22.844468 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:22.844468 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:22.844468 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:22.844468 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:22.846349 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:22.846349 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:22.846349 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:22.846349 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:22.846349 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:22.846349 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:22.846349 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:22.846349 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:22.846349 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:22.846349 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:22.846349 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:22.846349 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:22.846349 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:22.846349 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:22.846349 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:22.846349 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:22.846349 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:22.848178 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:22.848178 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:22.848178 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:22.848178 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 335/651] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:22.848178 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7c2e0-e6f5-4660-8a95-0320b1985259 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:22.864342 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-098d6587-3899-4ebf-8e60-78cd11a960bc None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 22 02:33:22.864342 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:22.864342 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:22.864342 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:22.864342 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:22.864342 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:22.864342 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:22.864342 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:22.864342 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:22.864342 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:22.864342 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:22.864342 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:22.864342 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:33:22.864342 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:33:22.864342 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:33:22.864342 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:33:22.867042 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:22.867042 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:22.867042 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:33:22.867042 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:22.867042 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:33:22.867042 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:22.867042 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:22.867042 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:22.867042 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:22.867042 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:22.867042 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:22.867042 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:22.867042 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:22.867042 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:22.867042 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:22.867042 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:22.867042 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:22.867042 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:22.869256 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:22.869256 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:22.869256 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:33:22.869256 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:33:22.869256 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:33:22.869256 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:33:22.869256 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:33:22.869256 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:33:22.869256 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 22 02:33:22.869256 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:22.870798 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 317/652] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Sep 22 02:33:22 2022] GET /identity/v3/users/neutron => generated 85 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:22.872580 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098d6587-3899-4ebf-8e60-78cd11a960bc None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:22.957635 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:23.015526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098d6587-3899-4ebf-8e60-78cd11a960bc None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:23.041276 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7c2e0-e6f5-4660-8a95-0320b1985259 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:23.067539 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9981c-3c5e-46e8-9ae2-b2f664e7fc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:23.068471 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9981c-3c5e-46e8-9ae2-b2f664e7fc05 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:23.068783 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9981c-3c5e-46e8-9ae2-b2f664e7fc05 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:23.069079 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9981c-3c5e-46e8-9ae2-b2f664e7fc05 None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:23.069760 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a9981c-3c5e-46e8-9ae2-b2f664e7fc05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:23.070067 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a9981c-3c5e-46e8-9ae2-b2f664e7fc05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:23.071526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a9981c-3c5e-46e8-9ae2-b2f664e7fc05 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:23.085551 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 318/653] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:23.086951 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9981c-3c5e-46e8-9ae2-b2f664e7fc05 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:23.106874 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f93eea-1336-437d-8a62-1fe43dc18fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:23.107587 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f93eea-1336-437d-8a62-1fe43dc18fae None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:23.107944 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f93eea-1336-437d-8a62-1fe43dc18fae None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:23.108289 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f93eea-1336-437d-8a62-1fe43dc18fae None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:23.109025 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f93eea-1336-437d-8a62-1fe43dc18fae None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:23.109418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f93eea-1336-437d-8a62-1fe43dc18fae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:23.110922 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f93eea-1336-437d-8a62-1fe43dc18fae None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:23.125627 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 336/654] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:23.130750 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f93eea-1336-437d-8a62-1fe43dc18fae None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:23.203588 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:23.265049 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9981c-3c5e-46e8-9ae2-b2f664e7fc05 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:23.296419 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:23.316533 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f93eea-1336-437d-8a62-1fe43dc18fae None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:23.326978 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c0ab28-bbeb-4758-bad1-3884174b34b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:23.327955 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0ab28-bbeb-4758-bad1-3884174b34b6 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:23.328435 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0ab28-bbeb-4758-bad1-3884174b34b6 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:23.328904 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0ab28-bbeb-4758-bad1-3884174b34b6 None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:23.330011 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c0ab28-bbeb-4758-bad1-3884174b34b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:23.330537 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c0ab28-bbeb-4758-bad1-3884174b34b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:23.332796 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c0ab28-bbeb-4758-bad1-3884174b34b6 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:23.355209 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 319/655] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:23.355209 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c0ab28-bbeb-4758-bad1-3884174b34b6 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:23.395627 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694caef-c58f-4f85-83fe-1c12ed81ef4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:23.397402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694caef-c58f-4f85-83fe-1c12ed81ef4d None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:23.398262 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694caef-c58f-4f85-83fe-1c12ed81ef4d None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:23.398870 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694caef-c58f-4f85-83fe-1c12ed81ef4d None admin] PATH_INFO: `/v3/users` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:23.400430 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c694caef-c58f-4f85-83fe-1c12ed81ef4d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:23.401162 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c694caef-c58f-4f85-83fe-1c12ed81ef4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:23.404166 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c694caef-c58f-4f85-83fe-1c12ed81ef4d None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:23.444883 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 337/656] 10.222.0.20 () {56 vars in 1139 bytes} [Thu Sep 22 02:33:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:23.446463 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694caef-c58f-4f85-83fe-1c12ed81ef4d None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:23.500221 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c0ab28-bbeb-4758-bad1-3884174b34b6 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:23.542735 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d3b3c-13ff-4f3b-a2b6-755dac0d3a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:23.544130 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d3b3c-13ff-4f3b-a2b6-755dac0d3a63 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:23.544602 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d3b3c-13ff-4f3b-a2b6-755dac0d3a63 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:23.545048 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d3b3c-13ff-4f3b-a2b6-755dac0d3a63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:23.552884 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4d3b3c-13ff-4f3b-a2b6-755dac0d3a63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:23.554929 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4d3b3c-13ff-4f3b-a2b6-755dac0d3a63 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:23.569457 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-5e4d3b3c-13ff-4f3b-a2b6-755dac0d3a63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:23.569457 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:23.569457 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:23.569457 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:23.569457 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:23.569457 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:23.569457 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:23.569457 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:23.569457 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:23.569457 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:23.569457 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:23.569457 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:23.569457 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:33:23.569457 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:33:23.569457 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:33:23.571538 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:33:23.571538 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:23.571538 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:23.571538 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:23.571538 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:23.571538 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:23.571538 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:23.571538 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:23.571538 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:23.571538 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:23.571538 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:23.571538 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:23.571538 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:23.571538 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:23.571538 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:23.571538 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:23.571538 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:23.573304 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:33:23.573304 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:33:23.573304 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:33:23.573304 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:33:23.573304 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:33:23.573304 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:33:23.573304 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:23.573304 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:23.573304 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 320/657] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:33:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:23.573304 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d3b3c-13ff-4f3b-a2b6-755dac0d3a63 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:23.616329 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:23.621649 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694caef-c58f-4f85-83fe-1c12ed81ef4d None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:23.624386 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:23.666127 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c294efd2-b7cb-4634-b878-0b7b1f1d6041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:23.666777 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294efd2-b7cb-4634-b878-0b7b1f1d6041 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:23.667094 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294efd2-b7cb-4634-b878-0b7b1f1d6041 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:23.667371 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294efd2-b7cb-4634-b878-0b7b1f1d6041 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:23.675452 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c294efd2-b7cb-4634-b878-0b7b1f1d6041 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:23.676959 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c294efd2-b7cb-4634-b878-0b7b1f1d6041 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:23.690487 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-c294efd2-b7cb-4634-b878-0b7b1f1d6041 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 02:33:23.690487 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:23.690487 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:23.690487 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:23.690487 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:23.690487 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:23.690487 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:23.690487 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:23.690487 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:23.690487 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:23.690487 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:23.690487 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:23.690487 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:33:23.690487 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:33:23.690487 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:33:23.690487 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:33:23.692392 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:23.692392 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:23.692392 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:33:23.692392 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:23.692392 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:33:23.692392 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:23.692392 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:23.692392 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:23.692392 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:23.692392 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:23.692392 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:23.692392 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:23.692392 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:23.692392 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:23.692392 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:23.692392 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:23.692392 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:23.692392 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:23.694449 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:23.694449 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:23.694449 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:33:23.694449 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:33:23.694449 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:33:23.694449 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:33:23.694449 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:33:23.694449 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:33:23.694449 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 02:33:23.694449 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:23.694449 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 338/658] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:33:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:23.694449 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c294efd2-b7cb-4634-b878-0b7b1f1d6041 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:23.712726 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d3b3c-13ff-4f3b-a2b6-755dac0d3a63 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:23.757345 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e9217-2156-463b-966b-c0dc16ed06f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:23.758008 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e9217-2156-463b-966b-c0dc16ed06f7 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:23.758289 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e9217-2156-463b-966b-c0dc16ed06f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:23.758562 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e9217-2156-463b-966b-c0dc16ed06f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:23.766044 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e9217-2156-463b-966b-c0dc16ed06f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:23.767714 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e9217-2156-463b-966b-c0dc16ed06f7 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:23.782418 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-180e9217-2156-463b-966b-c0dc16ed06f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:23.782418 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:23.782418 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:23.782418 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:23.782418 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:23.782418 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:23.782418 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:23.782418 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:23.782418 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:23.782418 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:23.782418 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:23.782418 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:23.782418 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:33:23.782418 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:33:23.782418 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:33:23.784247 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:33:23.784247 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:23.784247 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:23.784247 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:23.784247 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:23.784247 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:23.784247 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:23.784247 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:23.784247 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:23.784247 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:23.784247 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:23.784247 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:23.784247 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:23.784247 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:23.784247 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:23.784247 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:23.784247 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:23.786536 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:33:23.786536 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:33:23.786536 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:33:23.786536 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:33:23.786536 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:33:23.786536 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:33:23.786536 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:23.786536 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:23.786536 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 321/659] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:33:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:23.786536 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09adaf6a-4f44-4209-aee6-cd24f109fcf2 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:23.786536 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09adaf6a-4f44-4209-aee6-cd24f109fcf2 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:23.788892 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09adaf6a-4f44-4209-aee6-cd24f109fcf2 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:23.789945 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 322/660] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:33:23.792474 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09adaf6a-4f44-4209-aee6-cd24f109fcf2 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:23.825881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c294efd2-b7cb-4634-b878-0b7b1f1d6041 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:23.889306 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c410b9d-3c7b-4abd-8ee2-2feddb896c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:23.890213 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c410b9d-3c7b-4abd-8ee2-2feddb896c2c None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:23.890704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c410b9d-3c7b-4abd-8ee2-2feddb896c2c None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:23.891188 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c410b9d-3c7b-4abd-8ee2-2feddb896c2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:23.917900 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:23.927434 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09adaf6a-4f44-4209-aee6-cd24f109fcf2 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:23.931099 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:23.982395 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0800247-f3a7-4bb7-879d-16d44c981428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:23.983391 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0800247-f3a7-4bb7-879d-16d44c981428 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:23.983847 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0800247-f3a7-4bb7-879d-16d44c981428 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:23.984302 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0800247-f3a7-4bb7-879d-16d44c981428 None admin] PATH_INFO: `/v3/projects` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:23.985510 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0800247-f3a7-4bb7-879d-16d44c981428 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:23.986018 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0800247-f3a7-4bb7-879d-16d44c981428 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:23.988413 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0800247-f3a7-4bb7-879d-16d44c981428 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:23.997905 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c410b9d-3c7b-4abd-8ee2-2feddb896c2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:24.000689 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c410b9d-3c7b-4abd-8ee2-2feddb896c2c None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:24.017444 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 323/661] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Sep 22 02:33:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:24.017444 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0800247-f3a7-4bb7-879d-16d44c981428 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:24.139898 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 339/662] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:33:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 02:33:24.140798 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c410b9d-3c7b-4abd-8ee2-2feddb896c2c None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:24.149947 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0800247-f3a7-4bb7-879d-16d44c981428 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:24.205904 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:24.212583 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed403b56-b210-434e-9747-deaccebce7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:24.213846 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed403b56-b210-434e-9747-deaccebce7ba None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:24.214345 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed403b56-b210-434e-9747-deaccebce7ba None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:24.214957 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed403b56-b210-434e-9747-deaccebce7ba None admin] PATH_INFO: `/v3/users` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:24.216469 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed403b56-b210-434e-9747-deaccebce7ba None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:24.217037 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed403b56-b210-434e-9747-deaccebce7ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:24.220078 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed403b56-b210-434e-9747-deaccebce7ba None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:24.242135 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 324/663] 10.222.0.20 () {56 vars in 1137 bytes} [Thu Sep 22 02:33:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:24.243509 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376e7d5-48cb-4624-b571-340db4a8c6b6 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:24.243844 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376e7d5-48cb-4624-b571-340db4a8c6b6 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:24.244203 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376e7d5-48cb-4624-b571-340db4a8c6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:24.264234 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-9376e7d5-48cb-4624-b571-340db4a8c6b6 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:24.265523 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-9376e7d5-48cb-4624-b571-340db4a8c6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:24.266807 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c410b9d-3c7b-4abd-8ee2-2feddb896c2c None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:24.279254 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:24.310807 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e62d2b-8cfc-4065-8677-b3901e41f87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:24.311758 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e62d2b-8cfc-4065-8677-b3901e41f87e None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:24.312144 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e62d2b-8cfc-4065-8677-b3901e41f87e None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:24.312563 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e62d2b-8cfc-4065-8677-b3901e41f87e None admin] PATH_INFO: `/v3/projects` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:24.313470 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e62d2b-8cfc-4065-8677-b3901e41f87e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:24.313864 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e62d2b-8cfc-4065-8677-b3901e41f87e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:24.315624 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e62d2b-8cfc-4065-8677-b3901e41f87e None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:24.331456 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 340/664] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Sep 22 02:33:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:24.332354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e62d2b-8cfc-4065-8677-b3901e41f87e None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:24.368801 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 325/665] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:33:24.369694 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9376e7d5-48cb-4624-b571-340db4a8c6b6 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:24.519588 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9376e7d5-48cb-4624-b571-340db4a8c6b6 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:24.534977 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e62d2b-8cfc-4065-8677-b3901e41f87e None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:24.572384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:24.577627 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff312650-949b-4263-ad6d-62a4287bdb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:24.579372 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff312650-949b-4263-ad6d-62a4287bdb4f None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:24.579372 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff312650-949b-4263-ad6d-62a4287bdb4f None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:24.579648 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff312650-949b-4263-ad6d-62a4287bdb4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:24.597049 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff312650-949b-4263-ad6d-62a4287bdb4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:24.599667 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff312650-949b-4263-ad6d-62a4287bdb4f None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:24.600382 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae19dc0c-eb2c-4522-9747-9802b0eab6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:24.601338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae19dc0c-eb2c-4522-9747-9802b0eab6c8 None admin] REQUEST_METHOD: `PUT` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:24.601777 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae19dc0c-eb2c-4522-9747-9802b0eab6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:24.602217 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae19dc0c-eb2c-4522-9747-9802b0eab6c8 None admin] PATH_INFO: `/v3/projects/2a792f8dc45145d687ddd58c94d44b5e/users/4b9ad548622b4e54a6fecd0afd5ef3c2/roles/fb2fbee669e14ffbbe0aa70642eab1ce` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:24.604608 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:24.626843 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae19dc0c-eb2c-4522-9747-9802b0eab6c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a792f8dc45145d687ddd58c94d44b5e, user_id=4b9ad548622b4e54a6fecd0afd5ef3c2, role_id=fb2fbee669e14ffbbe0aa70642eab1ce)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:24.628820 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae19dc0c-eb2c-4522-9747-9802b0eab6c8 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:24.630651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-ff312650-949b-4263-ad6d-62a4287bdb4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:24.630651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:24.630651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:24.630651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:24.630651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:24.630651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:24.630651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:24.630651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:24.630651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:24.630651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:24.630651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:24.630651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:24.630651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:24.630651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:24.630651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:24.630651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:24.632991 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:24.632991 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:24.632991 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:24.632991 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:24.632991 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:24.632991 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:24.632991 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:24.632991 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:24.632991 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:24.632991 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:24.632991 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:24.632991 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:24.632991 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:24.632991 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:24.632991 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:24.632991 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:24.632991 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:24.634018 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:24.634018 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:24.634018 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:24.640329 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 326/666] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:24.642032 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff312650-949b-4263-ad6d-62a4287bdb4f None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:24.649314 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 341/667] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Sep 22 02:33:24 2022] PUT /identity/v3/projects/2a792f8dc45145d687ddd58c94d44b5e/users/4b9ad548622b4e54a6fecd0afd5ef3c2/roles/fb2fbee669e14ffbbe0aa70642eab1ce => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 02:33:24.649937 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae19dc0c-eb2c-4522-9747-9802b0eab6c8 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:24.820750 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae19dc0c-eb2c-4522-9747-9802b0eab6c8 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:24.839729 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff312650-949b-4263-ad6d-62a4287bdb4f None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:24.867612 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5136149-1942-4caa-bbe6-7a75a2d9f5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:24.868419 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5136149-1942-4caa-bbe6-7a75a2d9f5eb None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:24.868555 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5136149-1942-4caa-bbe6-7a75a2d9f5eb None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:24.868995 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5136149-1942-4caa-bbe6-7a75a2d9f5eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:24.869911 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5136149-1942-4caa-bbe6-7a75a2d9f5eb None admin] RBAC: Adding query filter params (role.id=fb2fbee669e14ffbbe0aa70642eab1ce, scope.project.id=2a792f8dc45145d687ddd58c94d44b5e, user.id=6160520562b74be3b25c49a30e990d8b) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:24.870225 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5136149-1942-4caa-bbe6-7a75a2d9f5eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:24.871781 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5136149-1942-4caa-bbe6-7a75a2d9f5eb None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:24.886862 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 342/668] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:33:24 2022] GET /identity/v3/role_assignments?user.id=6160520562b74be3b25c49a30e990d8b&scope.project.id=2a792f8dc45145d687ddd58c94d44b5e&role.id=fb2fbee669e14ffbbe0aa70642eab1ce => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:24.886862 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5136149-1942-4caa-bbe6-7a75a2d9f5eb None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:24.922066 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e24ed-a20e-4c3f-ada3-212ad66e0418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:24.923217 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e24ed-a20e-4c3f-ada3-212ad66e0418 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:24.923674 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e24ed-a20e-4c3f-ada3-212ad66e0418 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:24.924122 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e24ed-a20e-4c3f-ada3-212ad66e0418 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:24.950328 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:25.081046 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:25.099960 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88e24ed-a20e-4c3f-ada3-212ad66e0418 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:25.102713 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88e24ed-a20e-4c3f-ada3-212ad66e0418 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:25.117136 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5136149-1942-4caa-bbe6-7a75a2d9f5eb None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:25.170377 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eedb6eb-0769-4c58-b8a9-1ebb4d5ba8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:25.172228 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eedb6eb-0769-4c58-b8a9-1ebb4d5ba8eb None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:25.172228 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eedb6eb-0769-4c58-b8a9-1ebb4d5ba8eb None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:25.172228 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eedb6eb-0769-4c58-b8a9-1ebb4d5ba8eb None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:25.172228 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eedb6eb-0769-4c58-b8a9-1ebb4d5ba8eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:25.172734 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eedb6eb-0769-4c58-b8a9-1ebb4d5ba8eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:25.173915 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eedb6eb-0769-4c58-b8a9-1ebb4d5ba8eb None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:25.189183 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 343/669] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:25.189781 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eedb6eb-0769-4c58-b8a9-1ebb4d5ba8eb None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:25.306100 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 327/670] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:33:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 665 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 02:33:25.307029 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e24ed-a20e-4c3f-ada3-212ad66e0418 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:25.341152 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:25.360164 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eedb6eb-0769-4c58-b8a9-1ebb4d5ba8eb None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:25.403704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6918d25-9314-4f57-bbc3-2dccdcebf9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:25.404651 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6918d25-9314-4f57-bbc3-2dccdcebf9d6 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:25.405114 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6918d25-9314-4f57-bbc3-2dccdcebf9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:25.405548 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6918d25-9314-4f57-bbc3-2dccdcebf9d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:25.421439 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6918d25-9314-4f57-bbc3-2dccdcebf9d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:33:25.426810 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-f6918d25-9314-4f57-bbc3-2dccdcebf9d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:33:25.426810 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:25.426810 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:25.426810 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:25.426810 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:25.426810 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:25.426810 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:25.426810 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:25.426810 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:25.426810 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:25.426810 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:25.426810 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:25.426810 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:33:25.426810 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:33:25.426810 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:33:25.426810 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:33:25.427957 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:33:25.427957 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:33:25.427957 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:33:25.427957 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:33:25.427957 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:25.427957 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:25.427957 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:25.427957 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:25.427957 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:25.427957 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:25.427957 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:25.427957 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:25.427957 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:25.427957 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:25.427957 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:25.427957 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:25.429389 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:25.429389 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:25.429389 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:25.429389 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:25.429389 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:33:25.429389 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:33:25.429389 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:33:25.429389 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:33:25.429389 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:33:25.429389 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:33:25.429389 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:33:25.429389 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:25.432355 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 344/671] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Sep 22 02:33:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:25.433385 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6918d25-9314-4f57-bbc3-2dccdcebf9d6 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:25.479932 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:25.492150 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e24ed-a20e-4c3f-ada3-212ad66e0418 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:25.562925 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc1a44-ac02-46a6-b9fc-13ca546fe3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:25.564000 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc1a44-ac02-46a6-b9fc-13ca546fe3a2 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:25.564460 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc1a44-ac02-46a6-b9fc-13ca546fe3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:25.564943 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc1a44-ac02-46a6-b9fc-13ca546fe3a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:25.578312 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dc1a44-ac02-46a6-b9fc-13ca546fe3a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:25.581090 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dc1a44-ac02-46a6-b9fc-13ca546fe3a2 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:25.605099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-b2dc1a44-ac02-46a6-b9fc-13ca546fe3a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:25.605099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:25.605099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:25.605099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:25.605099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:25.605099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:25.605099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:25.605099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:25.605099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:25.605099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:25.605099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:25.605099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:25.605099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:33:25.605099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:33:25.605099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:33:25.606055 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:33:25.606055 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:25.606055 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:25.606055 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:25.606055 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:25.606055 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:25.606055 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:25.606055 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:25.606055 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:25.606055 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:25.606055 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:25.606055 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:25.606055 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:25.606055 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:25.606055 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:25.606055 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:25.606055 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:25.607678 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:33:25.607678 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:33:25.607678 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:33:25.607678 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:33:25.607678 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:33:25.607678 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:33:25.607678 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:25.607678 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:25.613208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 328/672] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:33:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:25.615114 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc1a44-ac02-46a6-b9fc-13ca546fe3a2 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:25.657553 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6918d25-9314-4f57-bbc3-2dccdcebf9d6 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:25.736757 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798facbd-937e-44cd-a8ce-4042e9529063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:25.738453 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798facbd-937e-44cd-a8ce-4042e9529063 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:25.739006 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798facbd-937e-44cd-a8ce-4042e9529063 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:25.739677 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798facbd-937e-44cd-a8ce-4042e9529063 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:25.761749 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798facbd-937e-44cd-a8ce-4042e9529063 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:25.764550 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798facbd-937e-44cd-a8ce-4042e9529063 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:25.796418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-798facbd-937e-44cd-a8ce-4042e9529063 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:25.796418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:25.796418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:25.796418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:25.796418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:25.796418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:25.796418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:25.796418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:25.796418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:25.796418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:25.796418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:25.796418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:25.796418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:25.796418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:25.796418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:25.796418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:25.798692 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:25.798692 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:25.798692 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:25.798692 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:25.798692 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:25.798692 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:25.798692 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:25.798692 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:25.798692 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:25.798692 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:25.798692 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:25.798692 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:25.798692 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:25.798692 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:25.798692 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:25.798692 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:25.798692 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:25.800797 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:25.800797 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:25.800797 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:25.800797 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 345/673] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:25.800797 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798facbd-937e-44cd-a8ce-4042e9529063 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:25.808422 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:25.823269 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc1a44-ac02-46a6-b9fc-13ca546fe3a2 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:25.892271 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db565b42-90a4-4636-ae0d-4c4d75694c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:25.893210 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db565b42-90a4-4636-ae0d-4c4d75694c69 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:25.893608 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db565b42-90a4-4636-ae0d-4c4d75694c69 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:25.894114 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db565b42-90a4-4636-ae0d-4c4d75694c69 None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:25.895396 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db565b42-90a4-4636-ae0d-4c4d75694c69 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:25.896186 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db565b42-90a4-4636-ae0d-4c4d75694c69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:25.899762 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db565b42-90a4-4636-ae0d-4c4d75694c69 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:25.923104 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 329/674] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:33:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:25.924640 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db565b42-90a4-4636-ae0d-4c4d75694c69 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:25.976040 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:25.994753 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798facbd-937e-44cd-a8ce-4042e9529063 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:26.068288 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7638bf8d-64bf-4922-b1e1-f9542199afc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:26.070596 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638bf8d-64bf-4922-b1e1-f9542199afc3 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:26.071092 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638bf8d-64bf-4922-b1e1-f9542199afc3 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:26.071540 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638bf8d-64bf-4922-b1e1-f9542199afc3 None admin] PATH_INFO: `/v3/projects` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:26.073001 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7638bf8d-64bf-4922-b1e1-f9542199afc3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:26.073707 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7638bf8d-64bf-4922-b1e1-f9542199afc3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:26.077596 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7638bf8d-64bf-4922-b1e1-f9542199afc3 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:26.096766 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db565b42-90a4-4636-ae0d-4c4d75694c69 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:26.112394 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 346/675] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Sep 22 02:33:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:26.120118 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7638bf8d-64bf-4922-b1e1-f9542199afc3 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:26.141803 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b8d1b-c460-4844-9e33-a284df03ff24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:26.143071 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b8d1b-c460-4844-9e33-a284df03ff24 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:26.143566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b8d1b-c460-4844-9e33-a284df03ff24 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:26.144059 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b8d1b-c460-4844-9e33-a284df03ff24 None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:26.144967 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4b8d1b-c460-4844-9e33-a284df03ff24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:26.145548 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4b8d1b-c460-4844-9e33-a284df03ff24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:26.147814 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4b8d1b-c460-4844-9e33-a284df03ff24 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:26.167359 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 330/676] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:26.168791 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b8d1b-c460-4844-9e33-a284df03ff24 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:26.243612 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:26.284558 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:26.288357 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7638bf8d-64bf-4922-b1e1-f9542199afc3 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:26.329670 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6f11a6-1c2b-470c-be3b-548f000c04d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:26.330339 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f11a6-1c2b-470c-be3b-548f000c04d8 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:26.330619 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f11a6-1c2b-470c-be3b-548f000c04d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:26.330928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f11a6-1c2b-470c-be3b-548f000c04d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:26.336944 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b8d1b-c460-4844-9e33-a284df03ff24 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:26.343166 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6f11a6-1c2b-470c-be3b-548f000c04d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:26.343881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6f11a6-1c2b-470c-be3b-548f000c04d8 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:26.360020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-ee6f11a6-1c2b-470c-be3b-548f000c04d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:26.360020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:26.360020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:26.360020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:26.360020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:26.360020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:26.360020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:26.360020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:26.360020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:26.360020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:26.360020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:26.360020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:26.360020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:26.360020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:26.360020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:26.360020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:26.361889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:26.361889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:26.361889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:26.361889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:26.361889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:26.361889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:26.361889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:26.361889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:26.361889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:26.361889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:26.361889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:26.361889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:26.361889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:26.361889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:26.361889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:26.361889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:26.361889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:26.363539 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:26.363539 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:26.363539 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:26.366376 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 347/677] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:26.367410 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6f11a6-1c2b-470c-be3b-548f000c04d8 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:26.413952 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea23a8-83f2-480d-8ef5-c9d1fa233dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:26.415993 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea23a8-83f2-480d-8ef5-c9d1fa233dfa None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:26.416785 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea23a8-83f2-480d-8ef5-c9d1fa233dfa None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:26.417608 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea23a8-83f2-480d-8ef5-c9d1fa233dfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:26.419637 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ea23a8-83f2-480d-8ef5-c9d1fa233dfa None admin] RBAC: Adding query filter params (role.id=fb2fbee669e14ffbbe0aa70642eab1ce, scope.project.id=2a792f8dc45145d687ddd58c94d44b5e, user.id=a0f635200ad241b3b31ed49701bc511b) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:26.420423 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ea23a8-83f2-480d-8ef5-c9d1fa233dfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:26.423849 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ea23a8-83f2-480d-8ef5-c9d1fa233dfa None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:26.437278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 331/678] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:33:26 2022] GET /identity/v3/role_assignments?user.id=a0f635200ad241b3b31ed49701bc511b&scope.project.id=2a792f8dc45145d687ddd58c94d44b5e&role.id=fb2fbee669e14ffbbe0aa70642eab1ce => generated 608 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:26.445669 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea23a8-83f2-480d-8ef5-c9d1fa233dfa None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:26.479995 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6f11a6-1c2b-470c-be3b-548f000c04d8 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:26.502786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:26.523269 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d92c98-ee8e-419c-8531-3ef648a8245c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:26.523910 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d92c98-ee8e-419c-8531-3ef648a8245c None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:26.524181 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d92c98-ee8e-419c-8531-3ef648a8245c None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:26.528870 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d92c98-ee8e-419c-8531-3ef648a8245c None admin] PATH_INFO: `/v3/projects/service` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:26.532072 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d92c98-ee8e-419c-8531-3ef648a8245c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:26.533546 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d92c98-ee8e-419c-8531-3ef648a8245c None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:26.546206 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-d9d92c98-ee8e-419c-8531-3ef648a8245c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:26.546206 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:26.546206 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:26.546206 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:26.546206 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:26.546206 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:26.546206 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:26.546206 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:26.546206 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:26.546206 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:26.546206 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:26.546206 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:26.546206 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:33:26.546206 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:33:26.546206 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:33:26.548005 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:33:26.548005 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:26.548005 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:26.548005 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:26.548005 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:26.548005 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:26.548005 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:26.548005 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:26.548005 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:26.548005 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:26.548005 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:26.548005 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:26.548005 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:26.548005 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:26.548005 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:26.548005 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:26.548005 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:26.549805 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:33:26.549805 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:33:26.549805 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:33:26.549805 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:33:26.549805 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:33:26.549805 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:33:26.549805 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:26.549805 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:26.552069 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 348/679] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:33:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:26.553954 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d92c98-ee8e-419c-8531-3ef648a8245c None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:26.707289 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea23a8-83f2-480d-8ef5-c9d1fa233dfa None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:26.755822 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb9e72-bca7-4b75-b8b3-63be7c42cc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:26.757133 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb9e72-bca7-4b75-b8b3-63be7c42cc66 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:26.757729 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb9e72-bca7-4b75-b8b3-63be7c42cc66 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:26.758231 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb9e72-bca7-4b75-b8b3-63be7c42cc66 None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:26.759184 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb9e72-bca7-4b75-b8b3-63be7c42cc66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:26.761338 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb9e72-bca7-4b75-b8b3-63be7c42cc66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:26.763177 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb9e72-bca7-4b75-b8b3-63be7c42cc66 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:26.786732 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 332/680] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:26.800639 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb9e72-bca7-4b75-b8b3-63be7c42cc66 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:26.801380 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d92c98-ee8e-419c-8531-3ef648a8245c None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:26.806419 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:26.897227 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482954d7-f263-4daf-bf32-6ff325a578e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:26.898606 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482954d7-f263-4daf-bf32-6ff325a578e0 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:26.899259 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482954d7-f263-4daf-bf32-6ff325a578e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:26.899801 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482954d7-f263-4daf-bf32-6ff325a578e0 None admin] PATH_INFO: `/v3/projects` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:26.900845 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482954d7-f263-4daf-bf32-6ff325a578e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:26.901435 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482954d7-f263-4daf-bf32-6ff325a578e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:26.906557 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482954d7-f263-4daf-bf32-6ff325a578e0 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:26.945507 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 349/681] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Sep 22 02:33:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:26.951867 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482954d7-f263-4daf-bf32-6ff325a578e0 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:26.985328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb9e72-bca7-4b75-b8b3-63be7c42cc66 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:27.026709 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:27.039006 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b201732-f23b-44c2-a944-656844a8dc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:27.041020 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b201732-f23b-44c2-a944-656844a8dc2f None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:27.041602 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b201732-f23b-44c2-a944-656844a8dc2f None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:27.042105 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b201732-f23b-44c2-a944-656844a8dc2f None admin] PATH_INFO: `/v3/users/nova` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:27.076071 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b201732-f23b-44c2-a944-656844a8dc2f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:27.079023 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b201732-f23b-44c2-a944-656844a8dc2f None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:27.107075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-3b201732-f23b-44c2-a944-656844a8dc2f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 02:33:27.107075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:27.107075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:27.107075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:27.107075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:27.107075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:27.107075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:27.107075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:27.107075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:27.107075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:27.107075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:27.107075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:27.107075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:33:27.107075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:33:27.107075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:33:27.107075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:33:27.108140 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:27.108140 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:27.108140 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:33:27.108140 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:27.108140 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:33:27.108140 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:27.108140 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:27.108140 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:27.108140 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:27.108140 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:27.108140 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:27.108140 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:27.108140 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:27.108140 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:27.108140 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:27.108140 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:27.108140 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:27.108140 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:27.109105 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:27.109105 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:27.109105 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:33:27.109105 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:33:27.109105 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:33:27.109105 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:33:27.109105 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:33:27.109105 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:33:27.109105 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 02:33:27.109105 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:27.114893 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 333/682] 10.222.0.20 () {56 vars in 1093 bytes} [Thu Sep 22 02:33:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:27.119872 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b201732-f23b-44c2-a944-656844a8dc2f None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:27.175893 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482954d7-f263-4daf-bf32-6ff325a578e0 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:27.243020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce730c04-5258-41fc-99a1-e5b283a942d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:27.243915 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce730c04-5258-41fc-99a1-e5b283a942d0 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:27.244452 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce730c04-5258-41fc-99a1-e5b283a942d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:27.244911 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce730c04-5258-41fc-99a1-e5b283a942d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:27.246247 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce730c04-5258-41fc-99a1-e5b283a942d0 None admin] RBAC: Adding query filter params (role.id=fb2fbee669e14ffbbe0aa70642eab1ce, scope.project.id=2a792f8dc45145d687ddd58c94d44b5e, user.id=a08a0ed98bc14328a3d24309d7996b51) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:27.246783 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce730c04-5258-41fc-99a1-e5b283a942d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:27.249407 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce730c04-5258-41fc-99a1-e5b283a942d0 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:27.266352 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:27.266655 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 350/683] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:33:26 2022] GET /identity/v3/role_assignments?user.id=a08a0ed98bc14328a3d24309d7996b51&scope.project.id=2a792f8dc45145d687ddd58c94d44b5e&role.id=fb2fbee669e14ffbbe0aa70642eab1ce => generated 608 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:33:27.336643 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b201732-f23b-44c2-a944-656844a8dc2f None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:27.402691 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b0eaf-3f9d-4456-a289-45a1b22272d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:27.404082 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b0eaf-3f9d-4456-a289-45a1b22272d7 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:27.404785 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b0eaf-3f9d-4456-a289-45a1b22272d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:27.405471 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b0eaf-3f9d-4456-a289-45a1b22272d7 None admin] PATH_INFO: `/v3/users` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:27.407096 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b0eaf-3f9d-4456-a289-45a1b22272d7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:27.407876 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b0eaf-3f9d-4456-a289-45a1b22272d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:27.410811 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b0eaf-3f9d-4456-a289-45a1b22272d7 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:27.443958 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 334/684] 10.222.0.20 () {56 vars in 1133 bytes} [Thu Sep 22 02:33:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:27.454518 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce730c04-5258-41fc-99a1-e5b283a942d0 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:27.715265 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:27.719637 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce730c04-5258-41fc-99a1-e5b283a942d0 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:27.793275 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adefc12e-c3d5-4ea5-bf5d-707100720423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:27.794473 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefc12e-c3d5-4ea5-bf5d-707100720423 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:27.795010 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefc12e-c3d5-4ea5-bf5d-707100720423 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:27.795498 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefc12e-c3d5-4ea5-bf5d-707100720423 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:27.870862 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518e9b3-9e6a-436b-93e8-cef42140a24a None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:27.871877 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518e9b3-9e6a-436b-93e8-cef42140a24a None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:27.872368 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518e9b3-9e6a-436b-93e8-cef42140a24a None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:27.874330 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 335/685] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:33:27.880375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27292b-9c99-453c-b81b-444b05077ce1 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:27.881068 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27292b-9c99-453c-b81b-444b05077ce1 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:27.881557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27292b-9c99-453c-b81b-444b05077ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:27.905732 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-7e27292b-9c99-453c-b81b-444b05077ce1 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:27.907309 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-7e27292b-9c99-453c-b81b-444b05077ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:27.932619 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adefc12e-c3d5-4ea5-bf5d-707100720423 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:27.935456 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adefc12e-c3d5-4ea5-bf5d-707100720423 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:27.996693 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 336/686] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 02:33:28.142105 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:28.149193 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 351/687] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:33:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 696 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 02:33:28.154309 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27292b-9c99-453c-b81b-444b05077ce1 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:28.241666 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:28.321763 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27292b-9c99-453c-b81b-444b05077ce1 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:28.373658 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae2c1c0-e64e-425b-bdc4-277a99c85b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:28.374327 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2c1c0-e64e-425b-bdc4-277a99c85b53 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:28.374608 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2c1c0-e64e-425b-bdc4-277a99c85b53 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:28.374900 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2c1c0-e64e-425b-bdc4-277a99c85b53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:28.391751 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae2c1c0-e64e-425b-bdc4-277a99c85b53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:28.393323 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae2c1c0-e64e-425b-bdc4-277a99c85b53 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:28.413637 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-1ae2c1c0-e64e-425b-bdc4-277a99c85b53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:28.413637 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:28.413637 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:28.413637 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:28.413637 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:28.413637 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:28.413637 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:28.413637 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:28.413637 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:28.413637 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:28.413637 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:28.413637 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:28.413637 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:28.413637 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:28.413637 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:28.413637 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:28.415526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:28.415526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:28.415526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:28.415526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:28.415526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:28.415526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:28.415526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:28.415526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:28.415526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:28.415526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:28.415526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:28.415526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:28.415526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:28.415526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:28.415526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:28.415526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:28.415526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:28.418223 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:28.418223 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:28.418223 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:28.418223 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 337/688] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:28.424717 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adefc12e-c3d5-4ea5-bf5d-707100720423 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:28.483266 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae2c1c0-e64e-425b-bdc4-277a99c85b53 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:28.652887 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adefc12e-c3d5-4ea5-bf5d-707100720423 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:28.673025 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae2c1c0-e64e-425b-bdc4-277a99c85b53 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:28.678887 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:28.734023 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f37c04c-ad0e-4093-b5a8-6bad9cbf247e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:28.734676 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37c04c-ad0e-4093-b5a8-6bad9cbf247e None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:28.735010 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37c04c-ad0e-4093-b5a8-6bad9cbf247e None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:28.735286 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37c04c-ad0e-4093-b5a8-6bad9cbf247e None admin] PATH_INFO: `/v3/roles/service` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:28.747934 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f37c04c-ad0e-4093-b5a8-6bad9cbf247e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:33:28.753146 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-0f37c04c-ad0e-4093-b5a8-6bad9cbf247e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:33:28.753146 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:28.753146 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:28.753146 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:28.753146 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:28.753146 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:28.753146 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:28.753146 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:28.753146 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:28.753146 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:28.753146 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:28.753146 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:28.753146 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:33:28.753146 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:33:28.753146 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:33:28.753146 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:33:28.755002 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:33:28.755002 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:33:28.755002 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:33:28.755002 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:33:28.755002 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:28.755002 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:28.755002 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:28.755002 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:28.755002 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:28.755002 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:28.755002 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:28.755002 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:28.755002 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:28.755002 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:28.755002 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:28.755002 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:28.756723 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:28.756723 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:28.756723 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:28.756723 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:28.756723 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:33:28.756723 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:33:28.756723 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:33:28.756723 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:33:28.756723 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:33:28.756723 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:33:28.756723 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:33:28.756723 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:28.756723 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 338/689] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Sep 22 02:33:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:33:28.758074 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67da0a6-93f9-4a3a-a4ad-bc04360d6ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:28.758074 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67da0a6-93f9-4a3a-a4ad-bc04360d6ab0 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:28.759314 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67da0a6-93f9-4a3a-a4ad-bc04360d6ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:28.759314 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67da0a6-93f9-4a3a-a4ad-bc04360d6ab0 None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:28.762852 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67da0a6-93f9-4a3a-a4ad-bc04360d6ab0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:28.762852 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67da0a6-93f9-4a3a-a4ad-bc04360d6ab0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:28.763219 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f37c04c-ad0e-4093-b5a8-6bad9cbf247e None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:28.772112 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67da0a6-93f9-4a3a-a4ad-bc04360d6ab0 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:28.826083 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 352/690] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:28.836000 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67da0a6-93f9-4a3a-a4ad-bc04360d6ab0 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:28.934036 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f37c04c-ad0e-4093-b5a8-6bad9cbf247e None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:28.961737 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:29.002784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e113570-be5b-464c-81fe-89fe3cbd0b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:29.003835 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e113570-be5b-464c-81fe-89fe3cbd0b2e None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:29.004312 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e113570-be5b-464c-81fe-89fe3cbd0b2e None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:29.004808 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e113570-be5b-464c-81fe-89fe3cbd0b2e None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:29.006623 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e113570-be5b-464c-81fe-89fe3cbd0b2e None admin] RBAC: Adding query filter params (name=service) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:29.007202 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e113570-be5b-464c-81fe-89fe3cbd0b2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:29.009668 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e113570-be5b-464c-81fe-89fe3cbd0b2e None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:29.039925 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 339/691] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Sep 22 02:33:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:33:29.042611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2348b2-cfa7-40ef-9e1d-e92a05e00a37 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:29.043339 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2348b2-cfa7-40ef-9e1d-e92a05e00a37 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:29.043985 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2348b2-cfa7-40ef-9e1d-e92a05e00a37 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:29.046860 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 340/692] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:33:29.048448 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2348b2-cfa7-40ef-9e1d-e92a05e00a37 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:29.053894 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67da0a6-93f9-4a3a-a4ad-bc04360d6ab0 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:29.102146 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9e9d1-3da2-42f1-8b1d-ea6a18f4f234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:29.103138 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9e9d1-3da2-42f1-8b1d-ea6a18f4f234 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:29.103568 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9e9d1-3da2-42f1-8b1d-ea6a18f4f234 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:29.103990 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9e9d1-3da2-42f1-8b1d-ea6a18f4f234 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:29.109621 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:29.117684 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b9e9d1-3da2-42f1-8b1d-ea6a18f4f234 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:29.121718 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b9e9d1-3da2-42f1-8b1d-ea6a18f4f234 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:29.148608 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-52b9e9d1-3da2-42f1-8b1d-ea6a18f4f234 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:29.148608 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:29.148608 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:29.148608 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:29.148608 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:29.148608 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:29.148608 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:29.148608 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:29.148608 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:29.148608 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:29.148608 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:29.148608 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:29.148608 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:33:29.148608 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:33:29.148608 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:33:29.151061 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:33:29.151061 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:29.151061 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:29.151061 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:29.151061 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:29.151061 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:29.151061 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:29.151061 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:29.151061 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:29.151061 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:29.151061 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:29.151061 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:29.151061 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:29.151061 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:29.151061 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:29.151061 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:29.151061 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:29.152816 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:33:29.152816 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:33:29.152816 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:33:29.152816 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:33:29.152816 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:33:29.152816 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:33:29.152816 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:29.152816 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:29.152816 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 353/693] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:33:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:33:29.152816 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccda012-ac48-452d-acf3-2e62cfdb7f87 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:29.152816 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccda012-ac48-452d-acf3-2e62cfdb7f87 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:29.153992 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccda012-ac48-452d-acf3-2e62cfdb7f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:29.194294 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-7ccda012-ac48-452d-acf3-2e62cfdb7f87 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:29.196996 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-7ccda012-ac48-452d-acf3-2e62cfdb7f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:29.234989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2348b2-cfa7-40ef-9e1d-e92a05e00a37 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:29.289345 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd22b88a-e205-4a98-ad48-0e5f0fccd638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:29.290251 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd22b88a-e205-4a98-ad48-0e5f0fccd638 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:29.290596 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd22b88a-e205-4a98-ad48-0e5f0fccd638 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:29.291076 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd22b88a-e205-4a98-ad48-0e5f0fccd638 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:29.307521 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd22b88a-e205-4a98-ad48-0e5f0fccd638 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:29.310068 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd22b88a-e205-4a98-ad48-0e5f0fccd638 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:29.333200 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-cd22b88a-e205-4a98-ad48-0e5f0fccd638 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:29.333200 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:29.333200 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:29.333200 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:29.333200 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:29.333200 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:29.333200 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:29.333200 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:29.333200 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:29.333200 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:29.333200 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:29.333200 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:29.333200 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:29.333200 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:29.333200 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:29.333200 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:29.335168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:29.335168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:29.335168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:29.335168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:29.335168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:29.335168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:29.335168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:29.335168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:29.335168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:29.335168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:29.335168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:29.335168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:29.335168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:29.335168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:29.335168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:29.335168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:29.335168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:29.337042 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:29.337042 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:29.337042 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:29.337042 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 341/694] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:29.337042 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd22b88a-e205-4a98-ad48-0e5f0fccd638 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:29.352123 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 354/695] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:33:29.353618 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccda012-ac48-452d-acf3-2e62cfdb7f87 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:29.413810 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:29.481829 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccda012-ac48-452d-acf3-2e62cfdb7f87 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:29.527560 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7b3c7-ff89-485d-8d70-2f7709712416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:29.527560 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7b3c7-ff89-485d-8d70-2f7709712416 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:29.532982 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7b3c7-ff89-485d-8d70-2f7709712416 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:29.532982 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7b3c7-ff89-485d-8d70-2f7709712416 None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:29.533664 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec7b3c7-ff89-485d-8d70-2f7709712416 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:29.533976 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec7b3c7-ff89-485d-8d70-2f7709712416 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:29.536588 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:29.537394 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec7b3c7-ff89-485d-8d70-2f7709712416 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:29.562455 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 355/696] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:29.570455 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7b3c7-ff89-485d-8d70-2f7709712416 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:29.589872 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd22b88a-e205-4a98-ad48-0e5f0fccd638 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:29.700907 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea77eed-cac3-43de-8b8f-fcb7dbf69339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:29.701995 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea77eed-cac3-43de-8b8f-fcb7dbf69339 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:29.702561 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea77eed-cac3-43de-8b8f-fcb7dbf69339 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:29.703096 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea77eed-cac3-43de-8b8f-fcb7dbf69339 None admin] PATH_INFO: `/v3/projects` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:29.704477 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea77eed-cac3-43de-8b8f-fcb7dbf69339 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:29.708777 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea77eed-cac3-43de-8b8f-fcb7dbf69339 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:29.711855 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea77eed-cac3-43de-8b8f-fcb7dbf69339 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:29.740097 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 342/697] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Sep 22 02:33:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:29.747259 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea77eed-cac3-43de-8b8f-fcb7dbf69339 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:29.832041 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7b3c7-ff89-485d-8d70-2f7709712416 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:29.845250 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:29.874589 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842faa74-1794-41e4-93ec-31d3e0ab8055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:29.875284 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842faa74-1794-41e4-93ec-31d3e0ab8055 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:29.875572 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842faa74-1794-41e4-93ec-31d3e0ab8055 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:29.875851 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842faa74-1794-41e4-93ec-31d3e0ab8055 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:29.886001 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842faa74-1794-41e4-93ec-31d3e0ab8055 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:29.889085 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842faa74-1794-41e4-93ec-31d3e0ab8055 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:29.914000 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-842faa74-1794-41e4-93ec-31d3e0ab8055 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 22 02:33:29.914000 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:29.914000 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:29.914000 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:29.914000 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:29.914000 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:29.914000 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:29.914000 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:29.914000 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:29.914000 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:29.914000 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:29.914000 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:29.914000 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:33:29.914000 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:33:29.914000 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:33:29.914000 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:33:29.915989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:29.915989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:29.915989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:33:29.915989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:29.915989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:33:29.915989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:29.915989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:29.915989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:29.915989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:29.915989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:29.915989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:29.915989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:29.915989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:29.915989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:29.915989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:29.915989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:29.915989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:29.915989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:29.917916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:29.917916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:29.917916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:33:29.917916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:33:29.917916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:33:29.917916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:33:29.917916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:33:29.917916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:33:29.917916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 22 02:33:29.917916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:29.917916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 356/698] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Sep 22 02:33:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:29.917916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842faa74-1794-41e4-93ec-31d3e0ab8055 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:29.938261 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea77eed-cac3-43de-8b8f-fcb7dbf69339 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:29.998559 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4445f50-e5d5-49dc-b878-5399b0c06b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:29.999443 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4445f50-e5d5-49dc-b878-5399b0c06b59 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:29.999910 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4445f50-e5d5-49dc-b878-5399b0c06b59 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:30.000366 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4445f50-e5d5-49dc-b878-5399b0c06b59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:30.001371 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4445f50-e5d5-49dc-b878-5399b0c06b59 None admin] RBAC: Adding query filter params (role.id=31a687548f55493b8ea666fe7e350303, scope.project.id=2a792f8dc45145d687ddd58c94d44b5e, user.id=a64ed6c8e15d479887ab1dc59997d715) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:30.001887 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4445f50-e5d5-49dc-b878-5399b0c06b59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:30.004389 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4445f50-e5d5-49dc-b878-5399b0c06b59 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:30.009434 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:30.015319 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 343/699] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:33:29 2022] GET /identity/v3/role_assignments?user.id=a64ed6c8e15d479887ab1dc59997d715&scope.project.id=2a792f8dc45145d687ddd58c94d44b5e&role.id=31a687548f55493b8ea666fe7e350303 => generated 259 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:33:30.016386 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8896cc3-8f58-4a0b-b50d-d251a4e3b9e2 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:30.016886 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8896cc3-8f58-4a0b-b50d-d251a4e3b9e2 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:30.017364 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8896cc3-8f58-4a0b-b50d-d251a4e3b9e2 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:30.027832 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 344/700] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:30 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:33:30.029301 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8896cc3-8f58-4a0b-b50d-d251a4e3b9e2 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:30.138034 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842faa74-1794-41e4-93ec-31d3e0ab8055 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:30.186447 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce861e-8cb1-4039-8c06-da98aa89a006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:30.187298 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce861e-8cb1-4039-8c06-da98aa89a006 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:30.187439 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce861e-8cb1-4039-8c06-da98aa89a006 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:30.187854 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce861e-8cb1-4039-8c06-da98aa89a006 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:30.205924 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce861e-8cb1-4039-8c06-da98aa89a006 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:30.208457 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce861e-8cb1-4039-8c06-da98aa89a006 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:30.235529 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-9dce861e-8cb1-4039-8c06-da98aa89a006 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:30.235529 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:30.235529 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:30.235529 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:30.235529 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:30.235529 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:30.235529 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:30.235529 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:30.235529 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:30.235529 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:30.235529 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:30.235529 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:30.235529 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:30.235529 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:30.235529 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:30.235529 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:30.237384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:30.237384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:30.237384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:30.237384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:30.237384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:30.237384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:30.237384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:30.237384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:30.237384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:30.237384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:30.237384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:30.237384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:30.237384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:30.237384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:30.237384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:30.237384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:30.237384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:30.239164 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:30.239164 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:30.239164 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:30.239164 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 357/701] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:30.239595 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0ccb8-55a3-485a-8f85-66ddd5d634a8 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:30.239595 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0ccb8-55a3-485a-8f85-66ddd5d634a8 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:30.239819 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0ccb8-55a3-485a-8f85-66ddd5d634a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:30.267981 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-55d0ccb8-55a3-485a-8f85-66ddd5d634a8 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:30.269436 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-55d0ccb8-55a3-485a-8f85-66ddd5d634a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:30.278885 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8896cc3-8f58-4a0b-b50d-d251a4e3b9e2 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:30.301458 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:30.341414 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccdeaf4-7cd9-4425-ba64-8391d86bbc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:30.343138 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccdeaf4-7cd9-4425-ba64-8391d86bbc68 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:30.343968 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccdeaf4-7cd9-4425-ba64-8391d86bbc68 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:30.344701 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccdeaf4-7cd9-4425-ba64-8391d86bbc68 None admin] PATH_INFO: `/v3/users` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:30.346282 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccdeaf4-7cd9-4425-ba64-8391d86bbc68 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:30.347118 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccdeaf4-7cd9-4425-ba64-8391d86bbc68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:30.350732 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccdeaf4-7cd9-4425-ba64-8391d86bbc68 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:30.388010 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 345/702] 10.222.0.20 () {56 vars in 1135 bytes} [Thu Sep 22 02:33:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:30.389618 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccdeaf4-7cd9-4425-ba64-8391d86bbc68 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:30.427068 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 358/703] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:33:30.434975 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0ccb8-55a3-485a-8f85-66ddd5d634a8 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:30.568959 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:30.632300 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccdeaf4-7cd9-4425-ba64-8391d86bbc68 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:30.677061 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11946cf6-f09a-44c9-8879-c48a76337b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:30.679322 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11946cf6-f09a-44c9-8879-c48a76337b41 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:30.679880 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11946cf6-f09a-44c9-8879-c48a76337b41 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:30.680374 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11946cf6-f09a-44c9-8879-c48a76337b41 None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:30.681324 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11946cf6-f09a-44c9-8879-c48a76337b41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:30.681887 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11946cf6-f09a-44c9-8879-c48a76337b41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:30.683546 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11946cf6-f09a-44c9-8879-c48a76337b41 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:30.699372 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 346/704] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:30.704835 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11946cf6-f09a-44c9-8879-c48a76337b41 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:30.751875 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0ccb8-55a3-485a-8f85-66ddd5d634a8 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:30.814085 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad0ca5a-34de-41b9-bbe0-893fd5319f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:30.815150 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad0ca5a-34de-41b9-bbe0-893fd5319f23 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:30.815637 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad0ca5a-34de-41b9-bbe0-893fd5319f23 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:30.816120 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad0ca5a-34de-41b9-bbe0-893fd5319f23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:30.853750 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:30.872128 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11946cf6-f09a-44c9-8879-c48a76337b41 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:30.907913 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:30.934444 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad0ca5a-34de-41b9-bbe0-893fd5319f23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:30.935989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad0ca5a-34de-41b9-bbe0-893fd5319f23 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:30.942185 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30751ef-dfab-427a-86bc-af8770844dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:30.943194 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30751ef-dfab-427a-86bc-af8770844dc4 None admin] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:30.943705 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30751ef-dfab-427a-86bc-af8770844dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:30.944155 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30751ef-dfab-427a-86bc-af8770844dc4 None admin] PATH_INFO: `/v3/users` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:30.945233 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30751ef-dfab-427a-86bc-af8770844dc4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:30.947898 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30751ef-dfab-427a-86bc-af8770844dc4 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:30.980024 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 347/705] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Sep 22 02:33:30 2022] POST /identity/v3/users => generated 254 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:33:30.981480 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c0c73-1b4f-43b5-87b6-4dc78c1f097c None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:30.981808 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c0c73-1b4f-43b5-87b6-4dc78c1f097c None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:30.982063 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c0c73-1b4f-43b5-87b6-4dc78c1f097c None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:30.983841 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 348/706] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:33:30.988908 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c0c73-1b4f-43b5-87b6-4dc78c1f097c None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:31.139279 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 359/707] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:33:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 702 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 02:33:31.139279 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aec611-4ca4-45f1-b0fe-6d666fcc73e7 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:31.139279 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aec611-4ca4-45f1-b0fe-6d666fcc73e7 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:31.139279 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aec611-4ca4-45f1-b0fe-6d666fcc73e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:31.183247 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-f4aec611-4ca4-45f1-b0fe-6d666fcc73e7 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:31.186279 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-f4aec611-4ca4-45f1-b0fe-6d666fcc73e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:31.212099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c0c73-1b4f-43b5-87b6-4dc78c1f097c None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:31.269372 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817e237-82c6-4e1c-88a1-7a875e8e0be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:31.270849 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817e237-82c6-4e1c-88a1-7a875e8e0be3 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:31.271354 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817e237-82c6-4e1c-88a1-7a875e8e0be3 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:31.272385 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817e237-82c6-4e1c-88a1-7a875e8e0be3 None admin] PATH_INFO: `/v3/services/network` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:31.273695 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817e237-82c6-4e1c-88a1-7a875e8e0be3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:31.277797 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817e237-82c6-4e1c-88a1-7a875e8e0be3 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:31.284000 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:31.298100 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-e817e237-82c6-4e1c-88a1-7a875e8e0be3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 02:33:31.298100 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:31.298100 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 02:33:31.298100 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 02:33:31.298100 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 02:33:31.298100 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 02:33:31.298100 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 02:33:31.298100 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:33:31.298100 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 02:33:31.298100 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:31.298100 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 02:33:31.298100 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:31.298100 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:31.298100 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:31.298100 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:31.298100 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:31.298100 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:31.299278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:31.299278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:31.299278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:31.299278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:31.299278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:31.299278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:31.299278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 02:33:31.299278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 02:33:31.299278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 02:33:31.299278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 02:33:31.299278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:31.299278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:31.299278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:31.299278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:31.299278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:31.299278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:31.300182 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:31.300182 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:31.300182 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:31.300182 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:31.300182 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:31.300182 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:31.300182 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:31.300182 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:31.300182 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:31.300182 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:31.300182 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 02:33:31.300182 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:33:31.300182 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 02:33:31.300182 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:31.302149 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 349/708] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:33:30 2022] GET /identity/v3/services/network => generated 88 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:31.302285 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817e237-82c6-4e1c-88a1-7a875e8e0be3 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:31.342853 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 360/709] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:33:31.342853 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aec611-4ca4-45f1-b0fe-6d666fcc73e7 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:31.382587 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:31.520248 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817e237-82c6-4e1c-88a1-7a875e8e0be3 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:31.545831 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aec611-4ca4-45f1-b0fe-6d666fcc73e7 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:31.622813 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783e23d-3a16-4361-8f1e-3fb7044c1b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:31.623892 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783e23d-3a16-4361-8f1e-3fb7044c1b22 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:31.624384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783e23d-3a16-4361-8f1e-3fb7044c1b22 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:31.624966 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783e23d-3a16-4361-8f1e-3fb7044c1b22 None admin] PATH_INFO: `/v3/services` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:31.625819 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0783e23d-3a16-4361-8f1e-3fb7044c1b22 None admin] RBAC: Adding query filter params (name=network) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:31.626333 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0783e23d-3a16-4361-8f1e-3fb7044c1b22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:31.627967 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0783e23d-3a16-4361-8f1e-3fb7044c1b22 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:31.633740 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea1813-ad8d-4671-9370-61f384631587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:31.634357 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 361/710] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Sep 22 02:33:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:31.638941 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea1813-ad8d-4671-9370-61f384631587 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:31.638941 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea1813-ad8d-4671-9370-61f384631587 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:31.638941 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea1813-ad8d-4671-9370-61f384631587 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:31.640814 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783e23d-3a16-4361-8f1e-3fb7044c1b22 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:31.666104 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eea1813-ad8d-4671-9370-61f384631587 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:31.668098 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eea1813-ad8d-4671-9370-61f384631587 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:31.697577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-3eea1813-ad8d-4671-9370-61f384631587 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:31.697577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:31.697577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:31.697577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:31.697577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:31.697577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:31.697577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:31.697577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:31.697577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:31.697577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:31.697577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:31.697577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:31.697577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:31.697577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:31.697577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:31.697577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:31.698622 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:31.698622 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:31.698622 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:31.698622 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:31.698622 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:31.698622 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:31.698622 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:31.698622 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:31.698622 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:31.698622 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:31.698622 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:31.698622 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:31.698622 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:31.698622 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:31.698622 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:31.698622 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:31.698622 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:31.699609 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:31.699609 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:31.699609 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:31.703030 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 350/711] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:31.712697 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea1813-ad8d-4671-9370-61f384631587 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:31.762227 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:31.843849 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783e23d-3a16-4361-8f1e-3fb7044c1b22 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:31.885735 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:31.890684 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7137ed50-bee2-4479-8880-e74cfacaa7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:31.891448 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7137ed50-bee2-4479-8880-e74cfacaa7cf None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:31.891801 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7137ed50-bee2-4479-8880-e74cfacaa7cf None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:31.893796 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7137ed50-bee2-4479-8880-e74cfacaa7cf None admin] PATH_INFO: `/v3/services` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:31.895441 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7137ed50-bee2-4479-8880-e74cfacaa7cf None admin] RBAC: Adding query filter params (type=network) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:31.895779 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7137ed50-bee2-4479-8880-e74cfacaa7cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:31.900930 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7137ed50-bee2-4479-8880-e74cfacaa7cf None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:31.910697 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea1813-ad8d-4671-9370-61f384631587 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:31.922992 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 362/712] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Sep 22 02:33:31 2022] GET /identity/v3/services?type=network => generated 124 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:31.924702 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7137ed50-bee2-4479-8880-e74cfacaa7cf None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:31.993125 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a9ace-06dc-4bd6-ba3d-40b82c9a3e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:31.995087 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a9ace-06dc-4bd6-ba3d-40b82c9a3e67 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:31.995760 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a9ace-06dc-4bd6-ba3d-40b82c9a3e67 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:31.996599 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a9ace-06dc-4bd6-ba3d-40b82c9a3e67 None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:31.997997 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a9ace-06dc-4bd6-ba3d-40b82c9a3e67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:31.998686 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a9ace-06dc-4bd6-ba3d-40b82c9a3e67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:32.001656 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a9ace-06dc-4bd6-ba3d-40b82c9a3e67 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:32.054938 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 351/713] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:33:32.059891 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a9ace-06dc-4bd6-ba3d-40b82c9a3e67 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:32.223728 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7137ed50-bee2-4479-8880-e74cfacaa7cf None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:32.249430 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:32.316294 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb7e90-643c-49dc-9e54-7b1ed271cae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:32.317520 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb7e90-643c-49dc-9e54-7b1ed271cae5 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:32.319384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb7e90-643c-49dc-9e54-7b1ed271cae5 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:32.319384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb7e90-643c-49dc-9e54-7b1ed271cae5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:32.319832 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cb7e90-643c-49dc-9e54-7b1ed271cae5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:32.322541 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cb7e90-643c-49dc-9e54-7b1ed271cae5 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:32.365865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-42cb7e90-643c-49dc-9e54-7b1ed271cae5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 02:33:32.365865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:32.365865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 02:33:32.365865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 02:33:32.365865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 02:33:32.365865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 02:33:32.365865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 02:33:32.365865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:33:32.365865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 02:33:32.365865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:32.365865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 02:33:32.365865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:32.365865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:32.365865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:32.365865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:32.365865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:32.365865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:32.367881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:32.367881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:32.367881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:32.367881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:32.367881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:32.367881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:32.367881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 02:33:32.367881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 02:33:32.367881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 02:33:32.367881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 02:33:32.367881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:32.367881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:32.367881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:32.367881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:32.367881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:32.367881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:32.370401 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:32.370401 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:32.370401 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:32.370401 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:32.370401 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:32.370401 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:32.370401 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:32.370401 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:32.370401 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:32.370401 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:32.370401 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 02:33:32.370401 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:33:32.370401 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 02:33:32.370401 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:32.374921 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 363/714] 10.222.0.20 () {56 vars in 1129 bytes} [Thu Sep 22 02:33:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 451 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:32.378502 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb7e90-643c-49dc-9e54-7b1ed271cae5 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:32.410549 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a9ace-06dc-4bd6-ba3d-40b82c9a3e67 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:32.511731 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd950ede-007f-4974-aef0-c529c63d59bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:32.512304 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd950ede-007f-4974-aef0-c529c63d59bc None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:32.512591 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd950ede-007f-4974-aef0-c529c63d59bc None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:32.512872 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd950ede-007f-4974-aef0-c529c63d59bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:32.528001 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd950ede-007f-4974-aef0-c529c63d59bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:32.529503 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd950ede-007f-4974-aef0-c529c63d59bc None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:32.544361 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-dd950ede-007f-4974-aef0-c529c63d59bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:32.544361 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:32.544361 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:32.544361 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:32.544361 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:32.544361 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:32.544361 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:32.544361 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:32.544361 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:32.544361 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:32.544361 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:32.544361 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:32.544361 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:33:32.544361 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:33:32.544361 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:33:32.550847 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:33:32.550847 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:32.550847 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:32.550847 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:32.550847 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:32.550847 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:32.550847 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:32.550847 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:32.550847 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:32.550847 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:32.550847 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:32.550847 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:32.550847 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:32.550847 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:32.550847 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:32.550847 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:32.550847 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:32.552375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:33:32.552375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:33:32.552375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:33:32.552375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:33:32.552375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:33:32.552375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:33:32.552375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:32.552375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:32.552375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 352/715] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:33:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 487 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:33:32.552375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63560be3-7962-4681-af59-968ee00814a3 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:32.552375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63560be3-7962-4681-af59-968ee00814a3 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:32.558723 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63560be3-7962-4681-af59-968ee00814a3 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:32.558723 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 353/716] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:33:32.558723 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18e904-f596-4433-a65b-97e93fb5a61b None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:32.558723 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18e904-f596-4433-a65b-97e93fb5a61b None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:32.558723 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18e904-f596-4433-a65b-97e93fb5a61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:32.580377 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-ef18e904-f596-4433-a65b-97e93fb5a61b None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:32.582569 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-ef18e904-f596-4433-a65b-97e93fb5a61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:32.588575 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:32.593764 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb7e90-643c-49dc-9e54-7b1ed271cae5 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:32.637150 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec0849f-6c4c-44f5-80e1-ba5b6c74e523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:32.637782 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0849f-6c4c-44f5-80e1-ba5b6c74e523 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:32.638064 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0849f-6c4c-44f5-80e1-ba5b6c74e523 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:32.638343 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0849f-6c4c-44f5-80e1-ba5b6c74e523 None admin] PATH_INFO: `/v3/services` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:32.639022 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec0849f-6c4c-44f5-80e1-ba5b6c74e523 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:32.639336 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec0849f-6c4c-44f5-80e1-ba5b6c74e523 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:32.640784 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec0849f-6c4c-44f5-80e1-ba5b6c74e523 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:32.650705 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 364/717] 10.222.0.20 () {56 vars in 1124 bytes} [Thu Sep 22 02:33:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:32.651623 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec0849f-6c4c-44f5-80e1-ba5b6c74e523 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:32.683036 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 354/718] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 02:33:32.684000 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18e904-f596-4433-a65b-97e93fb5a61b None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:32.725031 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:32.874548 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18e904-f596-4433-a65b-97e93fb5a61b None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:32.878643 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec0849f-6c4c-44f5-80e1-ba5b6c74e523 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:32.919082 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1f23f-b12f-401a-b23a-96e7608700f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:32.919736 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1f23f-b12f-401a-b23a-96e7608700f2 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:32.920022 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1f23f-b12f-401a-b23a-96e7608700f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:32.920302 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1f23f-b12f-401a-b23a-96e7608700f2 None admin] PATH_INFO: `/v3/services` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:32.920976 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f1f23f-b12f-401a-b23a-96e7608700f2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:32.921283 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f1f23f-b12f-401a-b23a-96e7608700f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:32.922687 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f1f23f-b12f-401a-b23a-96e7608700f2 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:32.940700 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 355/719] 10.222.0.20 () {56 vars in 1124 bytes} [Thu Sep 22 02:33:32 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:32.961282 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380805e8-38d1-4f21-8d0d-ed7f1df713dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:32.961927 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380805e8-38d1-4f21-8d0d-ed7f1df713dd None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:32.962221 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380805e8-38d1-4f21-8d0d-ed7f1df713dd None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:32.962492 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380805e8-38d1-4f21-8d0d-ed7f1df713dd None admin] PATH_INFO: `/v3/projects` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:32.963276 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380805e8-38d1-4f21-8d0d-ed7f1df713dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:32.963590 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380805e8-38d1-4f21-8d0d-ed7f1df713dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:32.965114 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380805e8-38d1-4f21-8d0d-ed7f1df713dd None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:32.980015 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 365/720] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Sep 22 02:33:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:32.989857 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1f23f-b12f-401a-b23a-96e7608700f2 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:32.994683 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:33.009918 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380805e8-38d1-4f21-8d0d-ed7f1df713dd None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:33.197719 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:33.206027 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380805e8-38d1-4f21-8d0d-ed7f1df713dd None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:33.263702 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1f23f-b12f-401a-b23a-96e7608700f2 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:33.274912 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f0669e-6e6f-47e4-90f1-c3ee8a5e2685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:33.275982 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0669e-6e6f-47e4-90f1-c3ee8a5e2685 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:33.276443 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0669e-6e6f-47e4-90f1-c3ee8a5e2685 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:33.276888 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0669e-6e6f-47e4-90f1-c3ee8a5e2685 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:33.299919 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f0669e-6e6f-47e4-90f1-c3ee8a5e2685 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:33:33.311347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-52f0669e-6e6f-47e4-90f1-c3ee8a5e2685 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:33:33.311347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:33.311347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:33.311347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:33.311347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:33.311347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:33.311347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:33.311347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:33.311347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:33.311347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:33.311347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:33.311347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:33.311347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:33:33.311347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:33:33.311347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:33:33.311347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:33:33.312981 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:33:33.312981 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:33:33.312981 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:33:33.312981 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:33:33.312981 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:33.312981 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:33.312981 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:33.312981 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:33.312981 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:33.312981 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:33.312981 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:33.312981 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:33.312981 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:33.312981 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:33.312981 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:33.312981 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:33.314375 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:33.314375 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:33.314375 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:33.314375 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:33.314375 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:33:33.314375 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:33:33.314375 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:33:33.314375 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:33:33.314375 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:33:33.314375 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:33:33.314375 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:33:33.314375 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:33.314375 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 366/721] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Sep 22 02:33:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:33:33.320522 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f0669e-6e6f-47e4-90f1-c3ee8a5e2685 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:33.322408 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1e262-1fd0-4ec5-874b-852eec5f51af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:33.323153 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1e262-1fd0-4ec5-874b-852eec5f51af None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:33.324243 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1e262-1fd0-4ec5-874b-852eec5f51af None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:33.324524 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1e262-1fd0-4ec5-874b-852eec5f51af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:33.325306 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc1e262-1fd0-4ec5-874b-852eec5f51af None admin] RBAC: Adding query filter params (role.id=fb2fbee669e14ffbbe0aa70642eab1ce, scope.project.id=2a792f8dc45145d687ddd58c94d44b5e, user.id=4b9ad548622b4e54a6fecd0afd5ef3c2) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:33.325627 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc1e262-1fd0-4ec5-874b-852eec5f51af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:33.327162 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc1e262-1fd0-4ec5-874b-852eec5f51af None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:33.339707 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 356/722] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:33:32 2022] GET /identity/v3/role_assignments?user.id=4b9ad548622b4e54a6fecd0afd5ef3c2&scope.project.id=2a792f8dc45145d687ddd58c94d44b5e&role.id=fb2fbee669e14ffbbe0aa70642eab1ce => generated 608 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:33.447629 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f0669e-6e6f-47e4-90f1-c3ee8a5e2685 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:33.491082 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7667e5c3-fad6-4a98-8554-4033bf92bee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:33.491747 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7667e5c3-fad6-4a98-8554-4033bf92bee1 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:33.492301 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7667e5c3-fad6-4a98-8554-4033bf92bee1 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:33.492581 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7667e5c3-fad6-4a98-8554-4033bf92bee1 None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:33.493289 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7667e5c3-fad6-4a98-8554-4033bf92bee1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:33.493601 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7667e5c3-fad6-4a98-8554-4033bf92bee1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:33.496443 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7667e5c3-fad6-4a98-8554-4033bf92bee1 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:33.507380 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 367/723] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:33:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:33.514030 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1e262-1fd0-4ec5-874b-852eec5f51af None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:33.697424 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:33.779329 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1e262-1fd0-4ec5-874b-852eec5f51af None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:33.869839 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c8a815-d27b-4f12-a5bb-6a6068ffd802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:33.870509 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8a815-d27b-4f12-a5bb-6a6068ffd802 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:33.871077 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8a815-d27b-4f12-a5bb-6a6068ffd802 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:33.871364 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8a815-d27b-4f12-a5bb-6a6068ffd802 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:33.889282 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c8a815-d27b-4f12-a5bb-6a6068ffd802 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:33.890801 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c8a815-d27b-4f12-a5bb-6a6068ffd802 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:33.907768 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-35c8a815-d27b-4f12-a5bb-6a6068ffd802 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:33.907768 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:33.907768 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:33.907768 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:33.907768 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:33.907768 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:33.907768 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:33.907768 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:33.907768 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:33.907768 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:33.907768 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:33.907768 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:33.907768 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:33.907768 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:33.907768 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:33.907768 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:33.909557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:33.909557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:33.909557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:33.909557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:33.909557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:33.909557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:33.909557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:33.909557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:33.909557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:33.909557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:33.909557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:33.909557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:33.909557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:33.909557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:33.909557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:33.909557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:33.909557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:33.912575 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:33.912575 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:33.912575 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:33.912575 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 357/724] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:33:33.915804 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7667e5c3-fad6-4a98-8554-4033bf92bee1 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:33.949048 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:34.049494 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7667e5c3-fad6-4a98-8554-4033bf92bee1 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:34.086566 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1db5789-5ad8-4a9b-89e5-79157b73eee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:34.087223 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db5789-5ad8-4a9b-89e5-79157b73eee1 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:34.087492 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db5789-5ad8-4a9b-89e5-79157b73eee1 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:34.087772 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db5789-5ad8-4a9b-89e5-79157b73eee1 None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:34.088438 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1db5789-5ad8-4a9b-89e5-79157b73eee1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:34.088746 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1db5789-5ad8-4a9b-89e5-79157b73eee1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:34.090154 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1db5789-5ad8-4a9b-89e5-79157b73eee1 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:34.105450 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 368/725] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:33:34.111823 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c8a815-d27b-4f12-a5bb-6a6068ffd802 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:34.313279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c8a815-d27b-4f12-a5bb-6a6068ffd802 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:34.354068 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:34.381258 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806901c-db9e-4b8c-9926-d4d3545d93de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:34.382946 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806901c-db9e-4b8c-9926-d4d3545d93de None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:34.383747 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806901c-db9e-4b8c-9926-d4d3545d93de None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:34.384523 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806901c-db9e-4b8c-9926-d4d3545d93de None admin] PATH_INFO: `/v3/users/nova` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:34.401460 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1806901c-db9e-4b8c-9926-d4d3545d93de None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:34.407469 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1806901c-db9e-4b8c-9926-d4d3545d93de None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:34.436876 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-1806901c-db9e-4b8c-9926-d4d3545d93de None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 02:33:34.436876 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:34.436876 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:34.436876 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:34.436876 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:34.436876 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:34.436876 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:34.436876 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:34.436876 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:34.436876 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:34.436876 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:34.436876 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:34.436876 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:33:34.436876 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:33:34.436876 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:33:34.436876 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:33:34.438721 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:34.438721 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:34.438721 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:33:34.438721 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:34.438721 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:33:34.438721 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:34.438721 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:34.438721 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:34.438721 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:34.438721 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:34.438721 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:34.438721 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:34.438721 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:34.438721 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:34.438721 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:34.438721 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:34.438721 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:34.438721 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:34.441392 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:34.441392 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:34.441392 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:33:34.441392 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:33:34.441392 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:33:34.441392 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:33:34.441392 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:33:34.441392 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:33:34.441392 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 02:33:34.441392 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:34.446047 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 358/726] 10.222.0.20 () {56 vars in 1093 bytes} [Thu Sep 22 02:33:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:33:34.451428 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1db5789-5ad8-4a9b-89e5-79157b73eee1 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:34.591695 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2d069-3a25-49e8-8091-745b2818d09f None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:34.592133 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2d069-3a25-49e8-8091-745b2818d09f None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:34.592647 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2d069-3a25-49e8-8091-745b2818d09f None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:34.595556 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 359/727] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:33:34.608803 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:34.619170 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d46b1-410e-4c07-b2e8-9610979e53ed None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:34.619543 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d46b1-410e-4c07-b2e8-9610979e53ed None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:34.619736 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d46b1-410e-4c07-b2e8-9610979e53ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:34.628987 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1db5789-5ad8-4a9b-89e5-79157b73eee1 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:34.658110 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-469d46b1-410e-4c07-b2e8-9610979e53ed None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:34.659435 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-469d46b1-410e-4c07-b2e8-9610979e53ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:34.672308 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa7f70-0dd0-41e7-8e10-b9c771fff739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:34.673001 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa7f70-0dd0-41e7-8e10-b9c771fff739 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:34.673286 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa7f70-0dd0-41e7-8e10-b9c771fff739 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:34.673562 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa7f70-0dd0-41e7-8e10-b9c771fff739 None admin] PATH_INFO: `/v3/users` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:34.675031 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa7f70-0dd0-41e7-8e10-b9c771fff739 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:34.675350 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa7f70-0dd0-41e7-8e10-b9c771fff739 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:34.676942 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa7f70-0dd0-41e7-8e10-b9c771fff739 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:34.698252 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 369/728] 10.222.0.20 () {56 vars in 1133 bytes} [Thu Sep 22 02:33:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:34.710589 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa7f70-0dd0-41e7-8e10-b9c771fff739 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:34.831597 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa7f70-0dd0-41e7-8e10-b9c771fff739 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:34.864423 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 360/729] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:33:34.888832 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01fb102-6205-4e5a-be14-f675e9b89875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:34.889585 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01fb102-6205-4e5a-be14-f675e9b89875 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:34.889871 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01fb102-6205-4e5a-be14-f675e9b89875 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:34.890139 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01fb102-6205-4e5a-be14-f675e9b89875 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:34.903426 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:34.915944 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01fb102-6205-4e5a-be14-f675e9b89875 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:34.917649 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01fb102-6205-4e5a-be14-f675e9b89875 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:34.937910 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-d01fb102-6205-4e5a-be14-f675e9b89875 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:34.937910 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:34.937910 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:34.937910 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:34.937910 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:34.937910 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:34.937910 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:34.937910 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:34.937910 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:34.937910 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:34.937910 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:34.937910 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:34.937910 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:34.937910 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:34.937910 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:34.937910 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:34.939015 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:34.939015 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:34.939015 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:34.939015 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:34.939015 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:34.939015 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:34.939015 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:34.939015 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:34.939015 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:34.939015 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:34.939015 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:34.939015 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:34.939015 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:34.939015 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:34.939015 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:34.939015 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:34.939015 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:34.939949 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:34.939949 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:34.939949 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:34.939949 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 370/730] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:33:34.946102 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d46b1-410e-4c07-b2e8-9610979e53ed None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:35.047739 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8a2c5-25af-48be-b548-3cb8ae07e0db None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:35.048417 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8a2c5-25af-48be-b548-3cb8ae07e0db None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:35.048921 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8a2c5-25af-48be-b548-3cb8ae07e0db None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:35.050799 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 371/731] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:33:35.056728 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff8e9c-6b14-44f8-b12c-fa77ffce539a None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:35.057340 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff8e9c-6b14-44f8-b12c-fa77ffce539a None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:35.057818 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff8e9c-6b14-44f8-b12c-fa77ffce539a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:35.087312 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-a2ff8e9c-6b14-44f8-b12c-fa77ffce539a None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:35.088775 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-a2ff8e9c-6b14-44f8-b12c-fa77ffce539a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:35.169442 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d46b1-410e-4c07-b2e8-9610979e53ed None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:35.171080 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 372/732] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:33:35.190764 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:35.237287 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b1ffb-c746-448c-8072-1183a99307ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:35.237931 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b1ffb-c746-448c-8072-1183a99307ab None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:35.238221 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b1ffb-c746-448c-8072-1183a99307ab None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:35.238499 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b1ffb-c746-448c-8072-1183a99307ab None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:35.239198 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b1ffb-c746-448c-8072-1183a99307ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:35.239511 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b1ffb-c746-448c-8072-1183a99307ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:35.241465 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b1ffb-c746-448c-8072-1183a99307ab None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:35.262973 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 361/733] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:33:35.268922 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff8e9c-6b14-44f8-b12c-fa77ffce539a None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:35.373536 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b1ffb-c746-448c-8072-1183a99307ab None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:35.419840 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff8e9c-6b14-44f8-b12c-fa77ffce539a None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:35.450908 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:35.469704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80b514-ef1c-4075-9e40-0cb33612d21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:35.471322 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80b514-ef1c-4075-9e40-0cb33612d21c None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:35.471923 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80b514-ef1c-4075-9e40-0cb33612d21c None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:35.472427 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80b514-ef1c-4075-9e40-0cb33612d21c None admin] PATH_INFO: `/v3/projects/service` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:35.482716 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f80b514-ef1c-4075-9e40-0cb33612d21c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:35.484790 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f80b514-ef1c-4075-9e40-0cb33612d21c None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:35.505401 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-3f80b514-ef1c-4075-9e40-0cb33612d21c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:35.505401 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:35.505401 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:35.505401 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:35.505401 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:35.505401 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:35.505401 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:35.505401 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:35.505401 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:35.505401 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:35.505401 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:35.505401 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:35.505401 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:33:35.505401 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:33:35.505401 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:33:35.506441 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:33:35.506441 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:35.506441 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:35.506441 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:35.506441 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:35.506441 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:35.506441 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:35.506441 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:35.506441 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:35.506441 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:35.506441 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:35.506441 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:35.506441 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:35.506441 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:35.506441 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:35.506441 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:35.506441 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:35.507388 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:33:35.507388 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:33:35.507388 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:33:35.507388 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:33:35.507388 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:33:35.507388 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:33:35.507388 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:35.507388 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:35.509760 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 373/734] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:33:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:33:35.515941 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80b514-ef1c-4075-9e40-0cb33612d21c None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:35.615142 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b1ffb-c746-448c-8072-1183a99307ab None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:35.665975 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416eb81c-f220-4494-81d7-8a5088ed313f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:35.667217 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416eb81c-f220-4494-81d7-8a5088ed313f None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:35.667746 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416eb81c-f220-4494-81d7-8a5088ed313f None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:35.668255 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416eb81c-f220-4494-81d7-8a5088ed313f None admin] PATH_INFO: `/v3/roles/service` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:35.685047 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416eb81c-f220-4494-81d7-8a5088ed313f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:33:35.690570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-416eb81c-f220-4494-81d7-8a5088ed313f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:33:35.690570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:35.690570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:35.690570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:35.690570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:35.690570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:35.690570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:35.690570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:35.690570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:35.690570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:35.690570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:35.690570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:35.690570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:33:35.690570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:33:35.690570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:33:35.690570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:33:35.692407 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:33:35.692407 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:33:35.692407 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:33:35.692407 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:33:35.692407 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:35.692407 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:35.692407 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:35.692407 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:35.692407 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:35.692407 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:35.692407 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:35.692407 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:35.692407 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:35.692407 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:35.692407 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:35.692407 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:35.694680 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:35.694680 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:35.694680 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:35.694680 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:35.694680 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:33:35.694680 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:33:35.694680 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:33:35.694680 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:33:35.694680 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:33:35.694680 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:33:35.694680 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:33:35.694680 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:35.698286 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 362/735] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Sep 22 02:33:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:35.698912 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416eb81c-f220-4494-81d7-8a5088ed313f None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:35.735952 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:35.748317 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80b514-ef1c-4075-9e40-0cb33612d21c None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:35.813851 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0790a07e-efeb-4754-98fa-f7cdd1a9a1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:35.815432 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790a07e-efeb-4754-98fa-f7cdd1a9a1e1 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:35.816121 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416eb81c-f220-4494-81d7-8a5088ed313f None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:35.816517 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790a07e-efeb-4754-98fa-f7cdd1a9a1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:35.817333 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790a07e-efeb-4754-98fa-f7cdd1a9a1e1 None admin] PATH_INFO: `/v3/projects` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:35.819588 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0790a07e-efeb-4754-98fa-f7cdd1a9a1e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:35.820456 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0790a07e-efeb-4754-98fa-f7cdd1a9a1e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:35.823571 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0790a07e-efeb-4754-98fa-f7cdd1a9a1e1 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:35.848606 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 374/736] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Sep 22 02:33:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:35.850623 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0790a07e-efeb-4754-98fa-f7cdd1a9a1e1 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:35.855306 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4968e7-92b9-4886-86a8-435be0859be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:35.856041 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4968e7-92b9-4886-86a8-435be0859be6 None admin] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:35.856204 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4968e7-92b9-4886-86a8-435be0859be6 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:35.856488 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4968e7-92b9-4886-86a8-435be0859be6 None admin] PATH_INFO: `/v3/services` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:35.857231 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4968e7-92b9-4886-86a8-435be0859be6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:35.859915 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4968e7-92b9-4886-86a8-435be0859be6 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:35.877019 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 363/737] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Sep 22 02:33:35 2022] POST /identity/v3/services => generated 240 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:33:35.883983 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4968e7-92b9-4886-86a8-435be0859be6 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:35.944042 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:36.074986 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0790a07e-efeb-4754-98fa-f7cdd1a9a1e1 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:36.162345 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:36.180934 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df4ab4e-2259-47af-9069-7e6ebe8ed5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:36.182550 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4ab4e-2259-47af-9069-7e6ebe8ed5aa None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:36.183353 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4ab4e-2259-47af-9069-7e6ebe8ed5aa None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:36.184181 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4ab4e-2259-47af-9069-7e6ebe8ed5aa None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:36.185735 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df4ab4e-2259-47af-9069-7e6ebe8ed5aa None admin] RBAC: Adding query filter params (name=service) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:36.186563 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df4ab4e-2259-47af-9069-7e6ebe8ed5aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:36.189543 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df4ab4e-2259-47af-9069-7e6ebe8ed5aa None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:36.192135 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4968e7-92b9-4886-86a8-435be0859be6 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:36.208255 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 375/738] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Sep 22 02:33:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:36.210579 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c36e1-1036-4729-845b-d6ad8c235afb None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:36.211129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c36e1-1036-4729-845b-d6ad8c235afb None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:36.211619 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c36e1-1036-4729-845b-d6ad8c235afb None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:36.218265 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 376/739] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:33:36.218265 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c36e1-1036-4729-845b-d6ad8c235afb None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:36.251777 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08794afa-b99d-47ec-846a-0b3eaf203d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:36.252963 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08794afa-b99d-47ec-846a-0b3eaf203d06 None admin] REQUEST_METHOD: `PUT` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:36.253491 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08794afa-b99d-47ec-846a-0b3eaf203d06 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:36.253962 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08794afa-b99d-47ec-846a-0b3eaf203d06 None admin] PATH_INFO: `/v3/projects/2a792f8dc45145d687ddd58c94d44b5e/users/a64ed6c8e15d479887ab1dc59997d715/roles/31a687548f55493b8ea666fe7e350303` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:36.259516 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08794afa-b99d-47ec-846a-0b3eaf203d06 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a792f8dc45145d687ddd58c94d44b5e, user_id=a64ed6c8e15d479887ab1dc59997d715, role_id=31a687548f55493b8ea666fe7e350303)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:36.262296 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08794afa-b99d-47ec-846a-0b3eaf203d06 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:36.289271 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 364/740] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Sep 22 02:33:35 2022] PUT /identity/v3/projects/2a792f8dc45145d687ddd58c94d44b5e/users/a64ed6c8e15d479887ab1dc59997d715/roles/31a687548f55493b8ea666fe7e350303 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 02:33:36.291884 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a596bd-9d34-4d78-9865-d60783e9ca17 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:36.292390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a596bd-9d34-4d78-9865-d60783e9ca17 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:36.292960 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a596bd-9d34-4d78-9865-d60783e9ca17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:36.329037 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-15a596bd-9d34-4d78-9865-d60783e9ca17 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:36.331450 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-15a596bd-9d34-4d78-9865-d60783e9ca17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:36.362319 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:36.369534 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c36e1-1036-4729-845b-d6ad8c235afb None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:36.417389 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7ebcb-2f5e-4530-92fe-cbaafafef9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:36.418085 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7ebcb-2f5e-4530-92fe-cbaafafef9eb None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:36.418410 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7ebcb-2f5e-4530-92fe-cbaafafef9eb None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:36.418732 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7ebcb-2f5e-4530-92fe-cbaafafef9eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:36.432025 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c7ebcb-2f5e-4530-92fe-cbaafafef9eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:36.433682 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c7ebcb-2f5e-4530-92fe-cbaafafef9eb None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:36.451580 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-91c7ebcb-2f5e-4530-92fe-cbaafafef9eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:36.451580 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:36.451580 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:36.451580 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:36.451580 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:36.451580 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:36.451580 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:36.451580 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:36.451580 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:36.451580 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:36.451580 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:36.451580 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:36.451580 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:36.451580 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:36.451580 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:36.451580 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:36.453466 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:36.453466 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:36.453466 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:36.453466 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:36.453466 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:36.453466 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:36.453466 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:36.453466 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:36.453466 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:36.453466 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:36.453466 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:36.453466 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:36.453466 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:36.453466 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:36.453466 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:36.453466 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:36.453466 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:36.455262 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:36.455262 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:36.455262 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:36.455262 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 377/741] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:36.464345 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7ebcb-2f5e-4530-92fe-cbaafafef9eb None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:36.488860 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 365/742] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:36 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 02:33:36.603960 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7ebcb-2f5e-4530-92fe-cbaafafef9eb None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:36.645403 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8cc89-2006-4ba7-991d-82728817f0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:36.646078 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8cc89-2006-4ba7-991d-82728817f0c5 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:36.646378 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8cc89-2006-4ba7-991d-82728817f0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:36.646659 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8cc89-2006-4ba7-991d-82728817f0c5 None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:36.647387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b8cc89-2006-4ba7-991d-82728817f0c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:36.647700 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b8cc89-2006-4ba7-991d-82728817f0c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:36.649361 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b8cc89-2006-4ba7-991d-82728817f0c5 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:36.663830 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 378/743] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:36.669113 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a596bd-9d34-4d78-9865-d60783e9ca17 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:36.814737 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:36.842214 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a596bd-9d34-4d78-9865-d60783e9ca17 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:36.890629 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeec451-0bf6-4597-a83c-e188c63405cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:36.891320 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeec451-0bf6-4597-a83c-e188c63405cb None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:36.891608 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeec451-0bf6-4597-a83c-e188c63405cb None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:36.891877 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeec451-0bf6-4597-a83c-e188c63405cb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:36.905960 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeec451-0bf6-4597-a83c-e188c63405cb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:36.907534 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeec451-0bf6-4597-a83c-e188c63405cb None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:36.935916 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-afeec451-0bf6-4597-a83c-e188c63405cb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 02:33:36.935916 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:36.935916 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:36.935916 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:36.935916 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:36.935916 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:36.935916 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:36.935916 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:36.935916 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:36.935916 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:36.935916 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:36.935916 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:36.935916 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:33:36.935916 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:33:36.935916 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:33:36.935916 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:33:36.937730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:36.937730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:36.937730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:33:36.937730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:36.937730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:33:36.937730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:36.937730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:36.937730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:36.937730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:36.937730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:36.937730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:36.937730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:36.937730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:36.937730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:36.937730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:36.937730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:36.937730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:36.937730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:36.939625 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:36.939625 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:36.939625 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:33:36.939625 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:33:36.939625 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:33:36.939625 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:33:36.939625 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:33:36.939625 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:33:36.939625 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 02:33:36.939625 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:36.939625 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 366/744] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Sep 22 02:33:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:33:36.947184 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8cc89-2006-4ba7-991d-82728817f0c5 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:36.961189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:37.096754 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8cc89-2006-4ba7-991d-82728817f0c5 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:37.123932 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeec451-0bf6-4597-a83c-e188c63405cb None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:37.167205 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a794a-5fa2-4fae-b4de-7ac0dcb98dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:37.168804 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a794a-5fa2-4fae-b4de-7ac0dcb98dbe None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:37.169820 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a794a-5fa2-4fae-b4de-7ac0dcb98dbe None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:37.171323 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a794a-5fa2-4fae-b4de-7ac0dcb98dbe None admin] PATH_INFO: `/v3/users` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:37.172002 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a794a-5fa2-4fae-b4de-7ac0dcb98dbe None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:37.172311 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a794a-5fa2-4fae-b4de-7ac0dcb98dbe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:37.173775 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a794a-5fa2-4fae-b4de-7ac0dcb98dbe None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:37.194370 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 379/745] 10.222.0.20 () {56 vars in 1149 bytes} [Thu Sep 22 02:33:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:37.195883 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7867739-3d05-4377-97bf-40cb7b527994 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:37.196158 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7867739-3d05-4377-97bf-40cb7b527994 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:37.196434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7867739-3d05-4377-97bf-40cb7b527994 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:37.202884 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 380/746] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:33:37.202884 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7867739-3d05-4377-97bf-40cb7b527994 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:37.293714 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:37.329767 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeec451-0bf6-4597-a83c-e188c63405cb None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:37.338421 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7867739-3d05-4377-97bf-40cb7b527994 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:37.388596 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49904fa9-7907-401e-a687-62d6a8aa88f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:37.389169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49904fa9-7907-401e-a687-62d6a8aa88f9 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:37.389498 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49904fa9-7907-401e-a687-62d6a8aa88f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:37.389776 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49904fa9-7907-401e-a687-62d6a8aa88f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:37.400890 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d706d-6398-4a15-ae67-251f85fc3249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:37.402449 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d706d-6398-4a15-ae67-251f85fc3249 None admin] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:37.403310 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d706d-6398-4a15-ae67-251f85fc3249 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:37.404098 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d706d-6398-4a15-ae67-251f85fc3249 None admin] PATH_INFO: `/v3/services` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:37.405675 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d706d-6398-4a15-ae67-251f85fc3249 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:37.408653 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d706d-6398-4a15-ae67-251f85fc3249 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:37.429798 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:37.440895 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 367/747] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Sep 22 02:33:37 2022] POST /identity/v3/services => generated 251 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:33:37.444333 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357815c4-07a9-46a1-aa44-872d35f96cb9 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:37.445227 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357815c4-07a9-46a1-aa44-872d35f96cb9 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:37.446104 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357815c4-07a9-46a1-aa44-872d35f96cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:37.481071 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-357815c4-07a9-46a1-aa44-872d35f96cb9 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:37.483790 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-357815c4-07a9-46a1-aa44-872d35f96cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:37.540434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49904fa9-7907-401e-a687-62d6a8aa88f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:37.548246 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49904fa9-7907-401e-a687-62d6a8aa88f9 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:37.714569 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:37.717782 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 368/748] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:33:37.756213 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 381/749] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:33:37 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 02:33:37.762311 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357815c4-07a9-46a1-aa44-872d35f96cb9 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:37.892987 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357815c4-07a9-46a1-aa44-872d35f96cb9 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:37.927938 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:37.938901 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f6f90-f104-4458-acc7-e03f18fb6f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:37.939592 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f6f90-f104-4458-acc7-e03f18fb6f33 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:37.939880 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f6f90-f104-4458-acc7-e03f18fb6f33 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:37.940160 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f6f90-f104-4458-acc7-e03f18fb6f33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:37.958043 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f6f90-f104-4458-acc7-e03f18fb6f33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:37.961672 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f6f90-f104-4458-acc7-e03f18fb6f33 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:37.993510 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-4f3f6f90-f104-4458-acc7-e03f18fb6f33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:37.993510 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:37.993510 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:37.993510 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:37.993510 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:37.993510 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:37.993510 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:37.993510 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:37.993510 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:37.993510 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:37.993510 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:37.993510 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:37.993510 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:37.993510 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:37.993510 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:37.993510 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:37.995400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:37.995400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:37.995400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:37.995400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:37.995400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:37.995400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:37.995400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:37.995400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:37.995400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:37.995400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:37.995400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:37.995400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:37.995400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:37.995400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:37.995400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:37.995400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:37.995400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:37.997098 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:37.997098 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:37.997098 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:37.997098 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 369/750] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:38.002180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49904fa9-7907-401e-a687-62d6a8aa88f9 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:38.154311 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49904fa9-7907-401e-a687-62d6a8aa88f9 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:38.194252 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a25a4c-526e-42f3-838f-5b1cc6cf9678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:38.194915 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a25a4c-526e-42f3-838f-5b1cc6cf9678 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:38.195349 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a25a4c-526e-42f3-838f-5b1cc6cf9678 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:38.195691 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a25a4c-526e-42f3-838f-5b1cc6cf9678 None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:38.196480 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a25a4c-526e-42f3-838f-5b1cc6cf9678 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:38.196793 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a25a4c-526e-42f3-838f-5b1cc6cf9678 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:38.198337 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a25a4c-526e-42f3-838f-5b1cc6cf9678 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:38.213519 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 382/751] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:38.219855 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f6f90-f104-4458-acc7-e03f18fb6f33 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:38.334071 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a25a4c-526e-42f3-838f-5b1cc6cf9678 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:38.369687 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f6f90-f104-4458-acc7-e03f18fb6f33 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:38.400031 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:38.429384 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7480a463-acf5-4c43-b50a-33ffcc35d1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:38.430043 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480a463-acf5-4c43-b50a-33ffcc35d1fd None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:38.430324 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480a463-acf5-4c43-b50a-33ffcc35d1fd None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:38.430606 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480a463-acf5-4c43-b50a-33ffcc35d1fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:38.451736 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7480a463-acf5-4c43-b50a-33ffcc35d1fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:38.453218 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7480a463-acf5-4c43-b50a-33ffcc35d1fd None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:38.469915 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-7480a463-acf5-4c43-b50a-33ffcc35d1fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:38.469915 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:38.469915 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:38.469915 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:38.469915 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:38.469915 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:38.469915 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:38.469915 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:38.469915 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:38.469915 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:38.469915 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:38.469915 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:38.469915 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:33:38.469915 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:33:38.469915 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:33:38.471653 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:33:38.471653 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:38.471653 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:38.471653 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:38.471653 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:38.471653 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:38.471653 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:38.471653 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:38.471653 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:38.471653 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:38.471653 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:38.471653 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:38.471653 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:38.471653 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:38.471653 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:38.471653 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:38.471653 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:38.473406 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:33:38.473406 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:33:38.473406 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:33:38.473406 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:33:38.473406 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:33:38.473406 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:33:38.473406 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:38.473406 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:38.473406 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 370/752] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:33:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:38.477462 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7480a463-acf5-4c43-b50a-33ffcc35d1fd None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:38.511070 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a25a4c-526e-42f3-838f-5b1cc6cf9678 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:38.551013 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f204506-28e7-4ec5-9c0f-5aae8e356c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:38.551822 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f204506-28e7-4ec5-9c0f-5aae8e356c7b None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:38.552083 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f204506-28e7-4ec5-9c0f-5aae8e356c7b None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:38.552907 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f204506-28e7-4ec5-9c0f-5aae8e356c7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:38.564818 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f204506-28e7-4ec5-9c0f-5aae8e356c7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:33:38.573879 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:38.576547 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-0f204506-28e7-4ec5-9c0f-5aae8e356c7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:33:38.576547 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:38.576547 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:38.576547 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:38.576547 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:38.576547 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:38.576547 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:38.576547 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:38.576547 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:38.576547 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:38.576547 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:38.576547 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:38.576547 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:33:38.576547 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:33:38.576547 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:33:38.576547 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:33:38.579952 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:33:38.579952 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:33:38.579952 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:33:38.579952 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:33:38.579952 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:38.579952 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:38.579952 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:38.579952 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:38.579952 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:38.579952 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:38.579952 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:38.579952 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:38.579952 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:38.579952 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:38.579952 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:38.579952 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:38.582684 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:38.582684 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:38.582684 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:38.582684 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:38.582684 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:33:38.582684 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:33:38.582684 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:33:38.582684 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:33:38.582684 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:33:38.582684 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:33:38.582684 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:33:38.582684 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:38.582684 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 383/753] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Sep 22 02:33:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:33:38.590261 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f204506-28e7-4ec5-9c0f-5aae8e356c7b None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:38.707396 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:38.715312 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f204506-28e7-4ec5-9c0f-5aae8e356c7b None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:38.756017 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1bc4b3-52e5-429f-81b8-ed3557c852af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:38.756834 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bc4b3-52e5-429f-81b8-ed3557c852af None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:38.757221 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bc4b3-52e5-429f-81b8-ed3557c852af None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:38.757605 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bc4b3-52e5-429f-81b8-ed3557c852af None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:38.758402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1bc4b3-52e5-429f-81b8-ed3557c852af None admin] RBAC: Adding query filter params (name=admin) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:38.758854 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1bc4b3-52e5-429f-81b8-ed3557c852af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:38.759020 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7480a463-acf5-4c43-b50a-33ffcc35d1fd None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:38.766862 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1bc4b3-52e5-429f-81b8-ed3557c852af None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:38.774937 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 384/754] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:33:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:33:38.777718 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1bc4b3-52e5-429f-81b8-ed3557c852af None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:38.843040 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b84aa7-63e7-4d67-8924-7f84b4eb8f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:38.843565 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b84aa7-63e7-4d67-8924-7f84b4eb8f2f None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:38.843848 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b84aa7-63e7-4d67-8924-7f84b4eb8f2f None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:38.844122 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b84aa7-63e7-4d67-8924-7f84b4eb8f2f None admin] PATH_INFO: `/v3/projects` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:38.844832 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b84aa7-63e7-4d67-8924-7f84b4eb8f2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:38.845139 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b84aa7-63e7-4d67-8924-7f84b4eb8f2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:38.846852 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b84aa7-63e7-4d67-8924-7f84b4eb8f2f None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:38.867101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 371/755] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Sep 22 02:33:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:38.868607 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c058a4-e4f9-4957-9acf-3b6e736084ec None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:38.868890 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c058a4-e4f9-4957-9acf-3b6e736084ec None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:38.869161 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c058a4-e4f9-4957-9acf-3b6e736084ec None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:38.872501 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 372/756] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:33:38.873388 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c058a4-e4f9-4957-9acf-3b6e736084ec None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:38.902390 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1bc4b3-52e5-429f-81b8-ed3557c852af None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:38.948452 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d208e9-29f3-440e-b09f-3dcdd34167d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:38.949145 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d208e9-29f3-440e-b09f-3dcdd34167d1 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:38.949471 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d208e9-29f3-440e-b09f-3dcdd34167d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:38.949755 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d208e9-29f3-440e-b09f-3dcdd34167d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:38.963473 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d208e9-29f3-440e-b09f-3dcdd34167d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:38.965034 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d208e9-29f3-440e-b09f-3dcdd34167d1 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:38.983025 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-03d208e9-29f3-440e-b09f-3dcdd34167d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:38.983025 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:38.983025 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:38.983025 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:38.983025 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:38.983025 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:38.983025 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:38.983025 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:38.983025 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:38.983025 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:38.983025 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:38.983025 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:38.983025 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:38.983025 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:38.983025 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:38.983025 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:38.985200 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:38.985200 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:38.985200 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:38.985200 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:38.985200 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:38.985200 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:38.985200 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:38.985200 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:38.985200 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:38.985200 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:38.985200 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:38.985200 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:38.985200 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:38.985200 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:38.985200 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:38.985200 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:38.985200 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:38.987510 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:38.987510 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:38.987510 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:38.987510 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 385/757] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:38.988309 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c5a94-45be-434e-9fbd-6f101ba8ad8d None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:38.988559 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c5a94-45be-434e-9fbd-6f101ba8ad8d None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:38.988785 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c5a94-45be-434e-9fbd-6f101ba8ad8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:39.001805 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c058a4-e4f9-4957-9acf-3b6e736084ec None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:39.014422 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:39.016089 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:39.038448 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-fa2c5a94-45be-434e-9fbd-6f101ba8ad8d None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:39.041280 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-fa2c5a94-45be-434e-9fbd-6f101ba8ad8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:39.063733 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeecf22-2a70-4b2b-831e-b0470db7a45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:39.064793 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeecf22-2a70-4b2b-831e-b0470db7a45d None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:39.065914 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeecf22-2a70-4b2b-831e-b0470db7a45d None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:39.066492 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeecf22-2a70-4b2b-831e-b0470db7a45d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:39.067999 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeecf22-2a70-4b2b-831e-b0470db7a45d None admin] RBAC: Adding query filter params (role.id=fb2fbee669e14ffbbe0aa70642eab1ce, scope.project.id=2a792f8dc45145d687ddd58c94d44b5e, user.id=9821de2a961f4ddab2fca8e45b74e3db) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:39.068609 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeecf22-2a70-4b2b-831e-b0470db7a45d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:39.070875 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeecf22-2a70-4b2b-831e-b0470db7a45d None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:39.080878 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 373/758] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:33:38 2022] GET /identity/v3/role_assignments?user.id=9821de2a961f4ddab2fca8e45b74e3db&scope.project.id=2a792f8dc45145d687ddd58c94d44b5e&role.id=fb2fbee669e14ffbbe0aa70642eab1ce => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:39.082224 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeecf22-2a70-4b2b-831e-b0470db7a45d None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:39.267615 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 386/759] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:33:39.333045 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeecf22-2a70-4b2b-831e-b0470db7a45d None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:39.371350 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e631e-8556-4807-b277-198b54d56f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:39.372413 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e631e-8556-4807-b277-198b54d56f60 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:39.372413 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e631e-8556-4807-b277-198b54d56f60 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:39.372670 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e631e-8556-4807-b277-198b54d56f60 None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:39.373153 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e631e-8556-4807-b277-198b54d56f60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:39.373464 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e631e-8556-4807-b277-198b54d56f60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:39.374893 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e631e-8556-4807-b277-198b54d56f60 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:39.389157 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 374/760] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:39.398329 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c5a94-45be-434e-9fbd-6f101ba8ad8d None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:39.609555 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c5a94-45be-434e-9fbd-6f101ba8ad8d None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:39.666406 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:39.674763 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c2119-cb5c-4521-94e1-4271d2ec994e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:39.676315 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c2119-cb5c-4521-94e1-4271d2ec994e None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:39.677107 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c2119-cb5c-4521-94e1-4271d2ec994e None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:39.677722 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c2119-cb5c-4521-94e1-4271d2ec994e None admin] PATH_INFO: `/v3/users/swift` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:39.691795 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8c2119-cb5c-4521-94e1-4271d2ec994e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:39.695435 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8c2119-cb5c-4521-94e1-4271d2ec994e None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:39.716241 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-ed8c2119-cb5c-4521-94e1-4271d2ec994e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 22 02:33:39.716241 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:39.716241 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:39.716241 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:39.716241 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:39.716241 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:39.716241 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:39.716241 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:39.716241 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:39.716241 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:39.716241 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:39.716241 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:39.716241 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:33:39.716241 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:33:39.716241 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:33:39.716241 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:33:39.718850 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:39.718850 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:39.718850 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:33:39.718850 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:39.718850 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:33:39.718850 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:39.718850 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:39.718850 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:39.718850 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:39.718850 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:39.718850 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:39.718850 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:39.718850 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:39.718850 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:39.718850 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:39.718850 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:39.718850 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:39.718850 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:39.720787 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:39.720787 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:39.720787 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:33:39.720787 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:33:39.720787 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:33:39.720787 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:33:39.720787 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:33:39.720787 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:33:39.720787 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 22 02:33:39.720787 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:39.725555 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 387/761] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Sep 22 02:33:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:39.735429 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e631e-8556-4807-b277-198b54d56f60 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:39.786939 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:39.799637 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c2119-cb5c-4521-94e1-4271d2ec994e None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:39.923196 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e631e-8556-4807-b277-198b54d56f60 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:39.948862 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c2119-cb5c-4521-94e1-4271d2ec994e None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:39.964073 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db352c-b7a0-4cfd-8f4a-efb21f61fa68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:39.964699 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db352c-b7a0-4cfd-8f4a-efb21f61fa68 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:39.965008 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db352c-b7a0-4cfd-8f4a-efb21f61fa68 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:39.965275 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db352c-b7a0-4cfd-8f4a-efb21f61fa68 None admin] PATH_INFO: `/v3/users` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:39.966016 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db352c-b7a0-4cfd-8f4a-efb21f61fa68 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:39.966338 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db352c-b7a0-4cfd-8f4a-efb21f61fa68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:39.967835 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db352c-b7a0-4cfd-8f4a-efb21f61fa68 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:39.994017 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 375/762] 10.222.0.20 () {56 vars in 1135 bytes} [Thu Sep 22 02:33:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:39.998749 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc06cfd-3451-4b84-82bc-208f254b1a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:39.999532 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc06cfd-3451-4b84-82bc-208f254b1a17 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:39.999913 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc06cfd-3451-4b84-82bc-208f254b1a17 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:40.000274 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc06cfd-3451-4b84-82bc-208f254b1a17 None admin] PATH_INFO: `/v3/services/network` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:40.001063 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc06cfd-3451-4b84-82bc-208f254b1a17 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:40.001381 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdeb055-5031-4170-a004-539f7b2af961 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:40.001886 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdeb055-5031-4170-a004-539f7b2af961 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:40.002376 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdeb055-5031-4170-a004-539f7b2af961 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:40.003175 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc06cfd-3451-4b84-82bc-208f254b1a17 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:40.004502 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 376/763] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:33:40.011146 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdeb055-5031-4170-a004-539f7b2af961 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:40.028613 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-bdc06cfd-3451-4b84-82bc-208f254b1a17 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 02:33:40.028613 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:40.028613 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 02:33:40.028613 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 02:33:40.028613 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 02:33:40.028613 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 02:33:40.028613 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 02:33:40.028613 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:33:40.028613 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 02:33:40.028613 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:40.028613 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 02:33:40.028613 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:40.028613 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:40.028613 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:40.028613 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:40.028613 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:40.028613 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:40.029683 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:40.029683 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:40.029683 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:40.029683 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:40.029683 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:40.029683 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:40.029683 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 02:33:40.029683 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 02:33:40.029683 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 02:33:40.029683 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 02:33:40.029683 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:40.029683 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:40.029683 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:40.029683 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:40.029683 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:40.029683 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:40.031197 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:40.031197 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:40.031197 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:40.031197 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:40.031197 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:40.031197 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:40.031197 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:40.031197 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:40.031197 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:40.031197 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:40.031197 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 02:33:40.031197 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:33:40.031197 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 02:33:40.031197 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:40.031197 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 388/764] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:33:39 2022] GET /identity/v3/services/network => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:40.032099 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9607744-2c82-4a4a-8ba0-68aac252d4dc None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:40.032099 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9607744-2c82-4a4a-8ba0-68aac252d4dc None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:40.032099 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9607744-2c82-4a4a-8ba0-68aac252d4dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:40.060294 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-d9607744-2c82-4a4a-8ba0-68aac252d4dc None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:40.062071 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-d9607744-2c82-4a4a-8ba0-68aac252d4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:40.144431 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:40.179011 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 389/765] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:33:40.179011 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9607744-2c82-4a4a-8ba0-68aac252d4dc None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:40.213412 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:40.281139 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdeb055-5031-4170-a004-539f7b2af961 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:40.407941 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ead98-633f-4f25-b1e2-860223a98f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:40.410039 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ead98-633f-4f25-b1e2-860223a98f92 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:40.411988 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ead98-633f-4f25-b1e2-860223a98f92 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:40.414166 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ead98-633f-4f25-b1e2-860223a98f92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:40.485848 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9607744-2c82-4a4a-8ba0-68aac252d4dc None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:40.548924 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3aef8-6317-4807-888a-c3fccdc6210e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:40.550502 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3aef8-6317-4807-888a-c3fccdc6210e None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:40.551240 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3aef8-6317-4807-888a-c3fccdc6210e None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:40.551881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3aef8-6317-4807-888a-c3fccdc6210e None admin] PATH_INFO: `/v3/services` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:40.553501 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be3aef8-6317-4807-888a-c3fccdc6210e None admin] RBAC: Adding query filter params (name=network) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:40.554081 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be3aef8-6317-4807-888a-c3fccdc6210e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:40.556685 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be3aef8-6317-4807-888a-c3fccdc6210e None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:40.566950 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 390/766] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Sep 22 02:33:40 2022] GET /identity/v3/services?name=network => generated 124 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:40.568561 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3aef8-6317-4807-888a-c3fccdc6210e None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:40.581280 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1ead98-633f-4f25-b1e2-860223a98f92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:40.585078 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1ead98-633f-4f25-b1e2-860223a98f92 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:40.653770 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:40.719837 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:40.721398 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3aef8-6317-4807-888a-c3fccdc6210e None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:40.747963 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 377/767] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:33:40 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 742 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 02:33:40.750360 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ead98-633f-4f25-b1e2-860223a98f92 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:40.782473 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c138d71-3346-4489-94d4-960bc22ba547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:40.783463 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c138d71-3346-4489-94d4-960bc22ba547 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:40.783929 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c138d71-3346-4489-94d4-960bc22ba547 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:40.784319 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c138d71-3346-4489-94d4-960bc22ba547 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:40.801212 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c138d71-3346-4489-94d4-960bc22ba547 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:33:40.809356 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-6c138d71-3346-4489-94d4-960bc22ba547 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:33:40.809356 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:40.809356 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:40.809356 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:40.809356 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:40.809356 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:40.809356 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:40.809356 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:40.809356 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:40.809356 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:40.809356 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:40.809356 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:40.809356 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:33:40.809356 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:33:40.809356 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:33:40.809356 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:33:40.810561 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:33:40.810561 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:33:40.810561 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:33:40.810561 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:33:40.810561 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:40.810561 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:40.810561 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:40.810561 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:40.810561 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:40.810561 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:40.810561 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:40.810561 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:40.810561 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:40.810561 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:40.810561 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:40.810561 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:40.812050 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:40.812050 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:40.812050 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:40.812050 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:40.812050 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:33:40.812050 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:33:40.812050 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:33:40.812050 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:33:40.812050 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:33:40.812050 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:33:40.812050 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:33:40.812050 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:40.812050 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 391/768] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Sep 22 02:33:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:40.812050 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd038f-1d09-4872-ad69-9e4063c6f922 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:40.812875 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd038f-1d09-4872-ad69-9e4063c6f922 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:40.812875 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd038f-1d09-4872-ad69-9e4063c6f922 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:40.819182 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 392/769] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:33:40.821199 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd038f-1d09-4872-ad69-9e4063c6f922 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:40.934453 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ead98-633f-4f25-b1e2-860223a98f92 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:40.973724 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1b7c6-3349-4434-bb59-25b3af47a0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:40.974913 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1b7c6-3349-4434-bb59-25b3af47a0cd None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:40.975206 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1b7c6-3349-4434-bb59-25b3af47a0cd None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:40.975546 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1b7c6-3349-4434-bb59-25b3af47a0cd None admin] PATH_INFO: `/v3/services` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:40.976119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb1b7c6-3349-4434-bb59-25b3af47a0cd None admin] RBAC: Adding query filter params (type=network) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:40.976427 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb1b7c6-3349-4434-bb59-25b3af47a0cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:40.977834 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb1b7c6-3349-4434-bb59-25b3af47a0cd None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:40.977953 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd038f-1d09-4872-ad69-9e4063c6f922 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:40.983691 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 378/770] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Sep 22 02:33:40 2022] GET /identity/v3/services?type=network => generated 350 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:40.984553 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1b7c6-3349-4434-bb59-25b3af47a0cd None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:40.999222 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:41.037715 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74239032-0eef-4216-a0b1-61822095d24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:41.038366 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74239032-0eef-4216-a0b1-61822095d24d None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:41.038647 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74239032-0eef-4216-a0b1-61822095d24d None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:41.039135 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74239032-0eef-4216-a0b1-61822095d24d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:41.052793 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74239032-0eef-4216-a0b1-61822095d24d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:41.054283 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74239032-0eef-4216-a0b1-61822095d24d None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:41.069810 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:41.074129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-74239032-0eef-4216-a0b1-61822095d24d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:41.074129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:41.074129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:41.074129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:41.074129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:41.074129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:41.074129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:41.074129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:41.074129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:41.074129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:41.074129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:41.074129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:41.074129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:41.074129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:41.074129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:41.074129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:41.075565 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:41.075565 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:41.075565 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:41.075565 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:41.075565 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:41.075565 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:41.075565 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:41.075565 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:41.075565 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:41.075565 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:41.075565 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:41.075565 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:41.075565 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:41.075565 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:41.075565 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:41.075565 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:41.075565 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:41.076454 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:41.076454 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:41.076454 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:41.076454 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 393/771] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:41.077848 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e2aaf-c652-43c1-92cf-ca5dbd5d30ab None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:41.078207 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e2aaf-c652-43c1-92cf-ca5dbd5d30ab None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:41.078577 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e2aaf-c652-43c1-92cf-ca5dbd5d30ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:41.100579 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-7a7e2aaf-c652-43c1-92cf-ca5dbd5d30ab None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:41.101768 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-7a7e2aaf-c652-43c1-92cf-ca5dbd5d30ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:41.174711 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1b7c6-3349-4434-bb59-25b3af47a0cd None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:41.228424 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994890b8-6822-467d-8d79-fb4cfe04e0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:41.229461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994890b8-6822-467d-8d79-fb4cfe04e0c5 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:41.229744 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 394/772] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:33:41.229905 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994890b8-6822-467d-8d79-fb4cfe04e0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:41.230436 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994890b8-6822-467d-8d79-fb4cfe04e0c5 None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:41.231610 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994890b8-6822-467d-8d79-fb4cfe04e0c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:41.232035 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7e2aaf-c652-43c1-92cf-ca5dbd5d30ab None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:41.232192 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994890b8-6822-467d-8d79-fb4cfe04e0c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:41.233936 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994890b8-6822-467d-8d79-fb4cfe04e0c5 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:41.255781 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 379/773] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:33:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:41.257113 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994890b8-6822-467d-8d79-fb4cfe04e0c5 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:41.399968 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994890b8-6822-467d-8d79-fb4cfe04e0c5 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:41.408528 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:41.443250 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94ea58-0c37-4038-b58e-4622ca0a4532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:41.444264 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94ea58-0c37-4038-b58e-4622ca0a4532 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:41.444615 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94ea58-0c37-4038-b58e-4622ca0a4532 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:41.445107 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94ea58-0c37-4038-b58e-4622ca0a4532 None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:41.446211 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f94ea58-0c37-4038-b58e-4622ca0a4532 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:41.446729 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f94ea58-0c37-4038-b58e-4622ca0a4532 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:41.448493 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f94ea58-0c37-4038-b58e-4622ca0a4532 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:41.475118 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 380/774] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:41.476316 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94ea58-0c37-4038-b58e-4622ca0a4532 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:41.510298 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7e2aaf-c652-43c1-92cf-ca5dbd5d30ab None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:41.519306 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:41.565162 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f8604-0e22-419b-81a2-cfedeb016227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:41.565926 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f8604-0e22-419b-81a2-cfedeb016227 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:41.566208 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f8604-0e22-419b-81a2-cfedeb016227 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:41.566517 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f8604-0e22-419b-81a2-cfedeb016227 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:41.567272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f8604-0e22-419b-81a2-cfedeb016227 None admin] RBAC: Adding query filter params (interface=public, service_id=29ed233fc87643978adb658ded3e657d, region_id=RegionOne) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:41.567583 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f8604-0e22-419b-81a2-cfedeb016227 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:41.569031 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f8604-0e22-419b-81a2-cfedeb016227 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:41.577084 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 395/775] 10.222.0.20 () {56 vars in 1251 bytes} [Thu Sep 22 02:33:41 2022] GET /identity/v3/endpoints?service_id=29ed233fc87643978adb658ded3e657d&interface=public®ion_id=RegionOne => generated 194 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:41.582339 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f8604-0e22-419b-81a2-cfedeb016227 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:41.605369 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94ea58-0c37-4038-b58e-4622ca0a4532 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:41.646974 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37309a13-92da-45e6-9344-d4ae412f5732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:41.647899 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37309a13-92da-45e6-9344-d4ae412f5732 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:41.648184 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37309a13-92da-45e6-9344-d4ae412f5732 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:41.648454 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37309a13-92da-45e6-9344-d4ae412f5732 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:41.659144 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37309a13-92da-45e6-9344-d4ae412f5732 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:41.660611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37309a13-92da-45e6-9344-d4ae412f5732 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:41.676323 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-37309a13-92da-45e6-9344-d4ae412f5732 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:41.676323 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:41.676323 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:41.676323 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:41.676323 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:41.676323 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:41.676323 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:41.676323 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:41.676323 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:41.676323 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:41.676323 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:41.676323 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:41.676323 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:41.676323 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:41.676323 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:41.676323 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:41.677461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:41.677461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:41.677461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:41.677461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:41.677461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:41.677461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:41.677461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:41.677461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:41.677461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:41.677461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:41.677461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:41.677461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:41.677461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:41.677461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:41.677461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:41.677461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:41.677461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:41.678783 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:41.678783 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:41.678783 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:41.678783 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 381/776] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:41.678783 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37309a13-92da-45e6-9344-d4ae412f5732 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:41.700372 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:41.719090 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f8604-0e22-419b-81a2-cfedeb016227 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:41.770531 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ed565-a5d4-4e6b-bb1b-ee3b3c738d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:41.772043 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ed565-a5d4-4e6b-bb1b-ee3b3c738d0d None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:41.772043 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ed565-a5d4-4e6b-bb1b-ee3b3c738d0d None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:41.772250 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ed565-a5d4-4e6b-bb1b-ee3b3c738d0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:41.779688 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131ed565-a5d4-4e6b-bb1b-ee3b3c738d0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:41.781155 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131ed565-a5d4-4e6b-bb1b-ee3b3c738d0d None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:41.789006 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37309a13-92da-45e6-9344-d4ae412f5732 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:41.792729 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-131ed565-a5d4-4e6b-bb1b-ee3b3c738d0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:41.792729 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:41.792729 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:41.792729 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:41.792729 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:41.792729 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:41.792729 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:41.792729 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:41.792729 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:41.792729 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:41.792729 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:41.792729 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:41.792729 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:33:41.792729 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:33:41.792729 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:33:41.793611 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:33:41.793611 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:41.793611 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:41.793611 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:41.793611 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:41.793611 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:41.793611 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:41.793611 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:41.793611 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:41.793611 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:41.793611 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:41.793611 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:41.793611 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:41.793611 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:41.793611 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:41.793611 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:41.793611 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:41.794530 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:33:41.794530 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:33:41.794530 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:33:41.794530 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:33:41.794530 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:33:41.794530 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:33:41.794530 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:41.794530 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:41.794530 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 396/777] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:33:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:41.795143 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ed565-a5d4-4e6b-bb1b-ee3b3c738d0d None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:41.813509 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:41.834775 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d41c30-56b9-42b1-b9bc-cf2dba3f6f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:41.835447 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d41c30-56b9-42b1-b9bc-cf2dba3f6f55 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:41.835728 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d41c30-56b9-42b1-b9bc-cf2dba3f6f55 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:41.836004 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d41c30-56b9-42b1-b9bc-cf2dba3f6f55 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:41.836601 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d41c30-56b9-42b1-b9bc-cf2dba3f6f55 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:41.838019 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d41c30-56b9-42b1-b9bc-cf2dba3f6f55 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:41.847514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-61d41c30-56b9-42b1-b9bc-cf2dba3f6f55 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 02:33:41.847514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:41.847514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 02:33:41.847514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 02:33:41.847514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 02:33:41.847514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 02:33:41.847514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 02:33:41.847514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:33:41.847514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 02:33:41.847514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:41.847514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 02:33:41.847514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:41.847514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:41.847514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:41.847514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:41.847514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:41.847514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:41.849587 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:41.849587 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:41.849587 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:41.849587 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:41.849587 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:41.849587 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:41.849587 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 02:33:41.849587 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 02:33:41.849587 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 02:33:41.849587 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 02:33:41.849587 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:41.849587 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:41.849587 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:41.849587 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:41.849587 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:41.849587 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:41.851611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:41.851611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:41.851611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:41.851611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:41.851611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:41.851611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:41.851611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:41.851611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:41.851611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:41.851611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:41.851611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 02:33:41.851611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:33:41.851611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 02:33:41.851611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:41.851611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 382/778] 10.222.0.20 () {56 vars in 1129 bytes} [Thu Sep 22 02:33:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:41.854271 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a8317-3e3f-4957-8902-2246171cd7a5 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:41.854271 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a8317-3e3f-4957-8902-2246171cd7a5 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:41.854271 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a8317-3e3f-4957-8902-2246171cd7a5 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:41.854271 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 383/779] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:33:41.854271 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a8317-3e3f-4957-8902-2246171cd7a5 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:41.912992 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ed565-a5d4-4e6b-bb1b-ee3b3c738d0d None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:41.948483 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:41.957239 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d355888-0258-4966-a29e-1cdb608ba89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:41.958294 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d355888-0258-4966-a29e-1cdb608ba89c None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:41.959453 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d355888-0258-4966-a29e-1cdb608ba89c None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:41.959959 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d355888-0258-4966-a29e-1cdb608ba89c None admin] PATH_INFO: `/v3/services` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:41.961088 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d355888-0258-4966-a29e-1cdb608ba89c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:41.963632 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d355888-0258-4966-a29e-1cdb608ba89c None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:41.970482 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a8317-3e3f-4957-8902-2246171cd7a5 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:41.982889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 397/780] 10.222.0.20 () {56 vars in 1087 bytes} [Thu Sep 22 02:33:41 2022] GET /identity/v3/services => generated 772 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:41.982889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d355888-0258-4966-a29e-1cdb608ba89c None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:42.139661 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9ed3d-48f5-46a2-9502-cc7c8faaf76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:42.140317 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9ed3d-48f5-46a2-9502-cc7c8faaf76c None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:42.140597 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9ed3d-48f5-46a2-9502-cc7c8faaf76c None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:42.140875 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9ed3d-48f5-46a2-9502-cc7c8faaf76c None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:42.142333 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9ed3d-48f5-46a2-9502-cc7c8faaf76c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:42.142659 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9ed3d-48f5-46a2-9502-cc7c8faaf76c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:42.144204 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9ed3d-48f5-46a2-9502-cc7c8faaf76c None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:42.155231 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:42.167366 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 384/781] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:42.167366 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9ed3d-48f5-46a2-9502-cc7c8faaf76c None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:42.197344 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d355888-0258-4966-a29e-1cdb608ba89c None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:42.256237 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036705d-6694-4917-800e-798e0c59e526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:42.256862 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036705d-6694-4917-800e-798e0c59e526 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:42.257171 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036705d-6694-4917-800e-798e0c59e526 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:42.257681 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036705d-6694-4917-800e-798e0c59e526 None admin] PATH_INFO: `/v3/projects` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:42.258236 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a036705d-6694-4917-800e-798e0c59e526 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:42.258563 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a036705d-6694-4917-800e-798e0c59e526 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:42.260190 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a036705d-6694-4917-800e-798e0c59e526 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:42.282988 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 398/782] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Sep 22 02:33:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:42.282988 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aaef7f-a906-45d8-b403-2d3b3d77aa1f None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:42.282988 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aaef7f-a906-45d8-b403-2d3b3d77aa1f None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:42.282988 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aaef7f-a906-45d8-b403-2d3b3d77aa1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:42.304047 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-b1aaef7f-a906-45d8-b403-2d3b3d77aa1f None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:42.305319 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-b1aaef7f-a906-45d8-b403-2d3b3d77aa1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:42.354107 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9ed3d-48f5-46a2-9502-cc7c8faaf76c None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:42.413887 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da726a9-1761-4d88-b57b-303175109f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:42.415100 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da726a9-1761-4d88-b57b-303175109f63 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:42.415602 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da726a9-1761-4d88-b57b-303175109f63 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:42.416096 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da726a9-1761-4d88-b57b-303175109f63 None admin] PATH_INFO: `/v3/services` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:42.417319 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da726a9-1761-4d88-b57b-303175109f63 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:42.417860 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da726a9-1761-4d88-b57b-303175109f63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:42.420394 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da726a9-1761-4d88-b57b-303175109f63 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:42.438907 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 399/783] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:33:42.438907 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aaef7f-a906-45d8-b403-2d3b3d77aa1f None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:42.439214 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 385/784] 10.222.0.20 () {56 vars in 1124 bytes} [Thu Sep 22 02:33:42 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:42.439214 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da726a9-1761-4d88-b57b-303175109f63 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:42.452558 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:42.572338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:42.598520 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da726a9-1761-4d88-b57b-303175109f63 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:42.625408 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aaef7f-a906-45d8-b403-2d3b3d77aa1f None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:42.660756 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f2fb9-32c7-4eeb-91fa-1d0f24eccee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:42.662381 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f2fb9-32c7-4eeb-91fa-1d0f24eccee5 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:42.662841 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f2fb9-32c7-4eeb-91fa-1d0f24eccee5 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:42.663343 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f2fb9-32c7-4eeb-91fa-1d0f24eccee5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:42.664798 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f2fb9-32c7-4eeb-91fa-1d0f24eccee5 None admin] RBAC: Adding query filter params (role.id=31a687548f55493b8ea666fe7e350303, scope.project.id=2a792f8dc45145d687ddd58c94d44b5e, user.id=4b9ad548622b4e54a6fecd0afd5ef3c2) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:42.665413 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f2fb9-32c7-4eeb-91fa-1d0f24eccee5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:42.668142 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f2fb9-32c7-4eeb-91fa-1d0f24eccee5 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:42.670703 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c6e2b-3232-4ee4-9bd0-d265b1fbac89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:42.671814 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c6e2b-3232-4ee4-9bd0-d265b1fbac89 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:42.672255 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c6e2b-3232-4ee4-9bd0-d265b1fbac89 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:42.672682 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c6e2b-3232-4ee4-9bd0-d265b1fbac89 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:42.682875 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128c6e2b-3232-4ee4-9bd0-d265b1fbac89 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:42.683350 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 386/785] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:33:42 2022] GET /identity/v3/role_assignments?user.id=4b9ad548622b4e54a6fecd0afd5ef3c2&scope.project.id=2a792f8dc45145d687ddd58c94d44b5e&role.id=31a687548f55493b8ea666fe7e350303 => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:42.684847 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128c6e2b-3232-4ee4-9bd0-d265b1fbac89 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:42.684974 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f2fb9-32c7-4eeb-91fa-1d0f24eccee5 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:42.709354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-128c6e2b-3232-4ee4-9bd0-d265b1fbac89 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 02:33:42.709354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:42.709354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:42.709354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:42.709354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:42.709354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:42.709354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:42.709354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:42.709354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:42.709354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:42.709354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:42.709354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:42.709354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:33:42.709354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:33:42.709354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:33:42.709354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:33:42.710568 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:42.710568 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:42.710568 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:33:42.710568 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:42.710568 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:33:42.710568 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:42.710568 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:42.710568 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:42.710568 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:42.710568 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:42.710568 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:42.710568 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:42.710568 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:42.710568 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:42.710568 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:42.710568 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:42.710568 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:42.710568 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:42.711618 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:42.711618 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:42.711618 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:33:42.711618 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:33:42.711618 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:33:42.711618 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:33:42.711618 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:33:42.711618 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:33:42.711618 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 02:33:42.711618 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:42.718790 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 400/786] 10.222.0.20 () {56 vars in 1093 bytes} [Thu Sep 22 02:33:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:42.727918 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c6e2b-3232-4ee4-9bd0-d265b1fbac89 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:42.829955 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:42.875278 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c6e2b-3232-4ee4-9bd0-d265b1fbac89 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:42.879072 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f2fb9-32c7-4eeb-91fa-1d0f24eccee5 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:42.920813 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:42.929070 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5259a0b2-6200-402c-bce6-82fea2726ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:42.930135 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc016bd-5432-497a-946e-c415360418da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:42.930270 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5259a0b2-6200-402c-bce6-82fea2726ab1 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:42.930776 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc016bd-5432-497a-946e-c415360418da None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:42.931046 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5259a0b2-6200-402c-bce6-82fea2726ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:42.931149 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc016bd-5432-497a-946e-c415360418da None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:42.931363 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc016bd-5432-497a-946e-c415360418da None admin] PATH_INFO: `/v3/services` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:42.931821 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5259a0b2-6200-402c-bce6-82fea2726ab1 None admin] PATH_INFO: `/v3/users` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:42.932032 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc016bd-5432-497a-946e-c415360418da None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:42.932349 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc016bd-5432-497a-946e-c415360418da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:42.933200 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5259a0b2-6200-402c-bce6-82fea2726ab1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:42.933769 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc016bd-5432-497a-946e-c415360418da None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:42.933985 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5259a0b2-6200-402c-bce6-82fea2726ab1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:42.935894 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5259a0b2-6200-402c-bce6-82fea2726ab1 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:42.940137 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 387/787] 10.222.0.20 () {56 vars in 1124 bytes} [Thu Sep 22 02:33:42 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:42.941043 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc016bd-5432-497a-946e-c415360418da None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:42.960207 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 401/788] 10.222.0.20 () {56 vars in 1133 bytes} [Thu Sep 22 02:33:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:42.961650 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5259a0b2-6200-402c-bce6-82fea2726ab1 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:43.105628 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5259a0b2-6200-402c-bce6-82fea2726ab1 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:43.134246 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc016bd-5432-497a-946e-c415360418da None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:43.150414 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dcfc4f-5e2c-4271-8e80-233161524ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:43.151132 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dcfc4f-5e2c-4271-8e80-233161524ef6 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:43.151416 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dcfc4f-5e2c-4271-8e80-233161524ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:43.151688 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dcfc4f-5e2c-4271-8e80-233161524ef6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:43.152424 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dcfc4f-5e2c-4271-8e80-233161524ef6 None admin] RBAC: Adding query filter params (interface=public, service_id=def3f6d07f2e4cf5a543e5cb5806e170, region_id=RegionOne) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:43.152738 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dcfc4f-5e2c-4271-8e80-233161524ef6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:43.154184 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dcfc4f-5e2c-4271-8e80-233161524ef6 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:43.162394 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 402/789] 10.222.0.20 () {56 vars in 1251 bytes} [Thu Sep 22 02:33:42 2022] GET /identity/v3/endpoints?service_id=def3f6d07f2e4cf5a543e5cb5806e170&interface=public®ion_id=RegionOne => generated 194 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:43.163504 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dcfc4f-5e2c-4271-8e80-233161524ef6 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:43.244286 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:43.251439 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7791d694-e8c4-490b-8a6e-9dbf2242e281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:43.254992 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791d694-e8c4-490b-8a6e-9dbf2242e281 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:43.256732 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791d694-e8c4-490b-8a6e-9dbf2242e281 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:43.258035 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791d694-e8c4-490b-8a6e-9dbf2242e281 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:43.265719 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:43.295566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7791d694-e8c4-490b-8a6e-9dbf2242e281 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:33:43.312452 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dcfc4f-5e2c-4271-8e80-233161524ef6 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:43.312808 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-7791d694-e8c4-490b-8a6e-9dbf2242e281 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:33:43.312808 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:43.312808 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:43.312808 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:43.312808 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:43.312808 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:43.312808 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:43.312808 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:43.312808 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:43.312808 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:43.312808 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:43.312808 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:43.312808 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:33:43.312808 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:33:43.312808 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:33:43.312808 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:33:43.315272 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:33:43.315272 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:33:43.315272 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:33:43.315272 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:33:43.315272 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:43.315272 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:43.315272 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:43.315272 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:43.315272 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:43.315272 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:43.315272 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:43.315272 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:43.315272 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:43.315272 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:43.315272 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:43.315272 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:43.316938 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:43.316938 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:43.316938 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:43.316938 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:43.316938 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:33:43.316938 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:33:43.316938 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:33:43.316938 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:33:43.316938 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:33:43.316938 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:33:43.316938 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:33:43.316938 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:43.320673 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 388/790] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Sep 22 02:33:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:43.324236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7791d694-e8c4-490b-8a6e-9dbf2242e281 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:43.391247 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5250b7-6bb2-40e0-b6c5-cd2ab77f5c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:43.391770 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5250b7-6bb2-40e0-b6c5-cd2ab77f5c33 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:43.392046 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5250b7-6bb2-40e0-b6c5-cd2ab77f5c33 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:43.392317 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5250b7-6bb2-40e0-b6c5-cd2ab77f5c33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:43.501276 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5250b7-6bb2-40e0-b6c5-cd2ab77f5c33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:43.502678 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7791d694-e8c4-490b-8a6e-9dbf2242e281 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:43.507159 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5250b7-6bb2-40e0-b6c5-cd2ab77f5c33 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:43.545160 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08accc-21d3-49a7-843e-a8d640403771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:43.545861 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08accc-21d3-49a7-843e-a8d640403771 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:43.546146 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08accc-21d3-49a7-843e-a8d640403771 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:43.546419 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08accc-21d3-49a7-843e-a8d640403771 None admin] PATH_INFO: `/v3/services` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:43.547360 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08accc-21d3-49a7-843e-a8d640403771 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:43.548544 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08accc-21d3-49a7-843e-a8d640403771 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:43.555668 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 389/791] 10.222.0.20 () {56 vars in 1087 bytes} [Thu Sep 22 02:33:43 2022] GET /identity/v3/services => generated 772 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:43.556583 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08accc-21d3-49a7-843e-a8d640403771 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:43.648019 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:43.665678 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:43.759019 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08accc-21d3-49a7-843e-a8d640403771 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:43.774935 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 403/792] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:33:43 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 02:33:43.784546 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5250b7-6bb2-40e0-b6c5-cd2ab77f5c33 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:43.835958 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e84c11-7525-4e1c-a66d-d65cba5a4fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:43.836656 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e84c11-7525-4e1c-a66d-d65cba5a4fb0 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:43.836940 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e84c11-7525-4e1c-a66d-d65cba5a4fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:43.837282 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e84c11-7525-4e1c-a66d-d65cba5a4fb0 None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:43.838096 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e84c11-7525-4e1c-a66d-d65cba5a4fb0 None admin] RBAC: Adding query filter params (name=service) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:43.838361 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e84c11-7525-4e1c-a66d-d65cba5a4fb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:43.839791 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e84c11-7525-4e1c-a66d-d65cba5a4fb0 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:43.858889 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 390/793] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Sep 22 02:33:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:43.863613 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e84c11-7525-4e1c-a66d-d65cba5a4fb0 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:44.027702 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5250b7-6bb2-40e0-b6c5-cd2ab77f5c33 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:44.045729 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e84c11-7525-4e1c-a66d-d65cba5a4fb0 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:44.073653 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:44.093197 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d643db-bca3-44e4-9c81-82e6e98c281b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:44.093867 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d643db-bca3-44e4-9c81-82e6e98c281b None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:44.094150 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d643db-bca3-44e4-9c81-82e6e98c281b None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:44.094423 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d643db-bca3-44e4-9c81-82e6e98c281b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:44.102705 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883b32c-f03b-4d0c-9f3a-6b8944f37212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:44.104298 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883b32c-f03b-4d0c-9f3a-6b8944f37212 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:44.105087 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883b32c-f03b-4d0c-9f3a-6b8944f37212 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:44.105878 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883b32c-f03b-4d0c-9f3a-6b8944f37212 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:44.107818 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d643db-bca3-44e4-9c81-82e6e98c281b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:44.109323 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d643db-bca3-44e4-9c81-82e6e98c281b None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:44.128584 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-c6d643db-bca3-44e4-9c81-82e6e98c281b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:44.128584 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:44.128584 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:44.128584 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:44.128584 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:44.128584 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:44.128584 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:44.128584 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:44.128584 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:44.128584 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:44.128584 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:44.128584 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:44.128584 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:44.128584 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:44.128584 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:44.128584 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:44.131057 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:44.131057 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:44.131057 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:44.131057 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:44.131057 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:44.131057 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:44.131057 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:44.131057 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:44.131057 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:44.131057 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:44.131057 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:44.131057 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:44.131057 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:44.131057 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:44.131057 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:44.131057 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:44.131057 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:44.132852 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:44.132852 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:44.132852 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:44.132852 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 391/794] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:44.133302 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c883b32c-f03b-4d0c-9f3a-6b8944f37212 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:44.133302 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c883b32c-f03b-4d0c-9f3a-6b8944f37212 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:44.138983 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d643db-bca3-44e4-9c81-82e6e98c281b None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:44.165243 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-c883b32c-f03b-4d0c-9f3a-6b8944f37212 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:44.165243 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:44.165243 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:44.165243 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:44.165243 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:44.165243 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:44.165243 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:44.165243 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:44.165243 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:44.165243 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:44.165243 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:44.165243 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:44.165243 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:44.165243 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:44.165243 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:44.165243 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:44.167525 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:44.167525 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:44.167525 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:44.167525 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:44.167525 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:44.167525 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:44.167525 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:44.167525 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:44.167525 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:44.167525 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:44.167525 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:44.167525 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:44.167525 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:44.167525 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:44.167525 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:44.167525 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:44.167525 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:44.169345 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:44.169345 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:44.169345 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:44.172792 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 404/795] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:44.179881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883b32c-f03b-4d0c-9f3a-6b8944f37212 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:44.187500 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:44.431787 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d643db-bca3-44e4-9c81-82e6e98c281b None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:44.528969 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883b32c-f03b-4d0c-9f3a-6b8944f37212 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:44.551668 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fa301-3cca-4ce5-9b41-71890499b031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:44.552924 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fa301-3cca-4ce5-9b41-71890499b031 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:44.553195 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fa301-3cca-4ce5-9b41-71890499b031 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:44.553414 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fa301-3cca-4ce5-9b41-71890499b031 None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:44.554309 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78fa301-3cca-4ce5-9b41-71890499b031 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:44.554309 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78fa301-3cca-4ce5-9b41-71890499b031 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:44.555776 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78fa301-3cca-4ce5-9b41-71890499b031 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:44.578416 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 392/796] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 440 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:33:44.583879 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fa301-3cca-4ce5-9b41-71890499b031 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:44.604255 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6619d15b-1bb1-4506-bca4-259920607b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:44.605830 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619d15b-1bb1-4506-bca4-259920607b22 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:44.606660 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619d15b-1bb1-4506-bca4-259920607b22 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:44.607456 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619d15b-1bb1-4506-bca4-259920607b22 None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:44.609960 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6619d15b-1bb1-4506-bca4-259920607b22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:44.610808 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6619d15b-1bb1-4506-bca4-259920607b22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:44.615241 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6619d15b-1bb1-4506-bca4-259920607b22 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:44.641101 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 405/797] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 463 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:44.648789 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6619d15b-1bb1-4506-bca4-259920607b22 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:44.684423 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:44.781728 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:44.862460 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6619d15b-1bb1-4506-bca4-259920607b22 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:44.898433 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fa301-3cca-4ce5-9b41-71890499b031 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:44.916946 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b445b632-3047-4236-988f-1337162da417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:44.917586 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445b632-3047-4236-988f-1337162da417 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:44.917869 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445b632-3047-4236-988f-1337162da417 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:44.918142 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445b632-3047-4236-988f-1337162da417 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:44.925860 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b445b632-3047-4236-988f-1337162da417 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:44.927800 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b445b632-3047-4236-988f-1337162da417 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:44.941886 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-b445b632-3047-4236-988f-1337162da417 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:44.941886 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:44.941886 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:44.941886 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:44.941886 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:44.941886 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:44.941886 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:44.941886 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:44.941886 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:44.941886 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:44.941886 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:44.941886 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:44.941886 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:33:44.941886 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:33:44.941886 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:33:44.944104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:33:44.944104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:44.944104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:44.944104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:44.944104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:44.944104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:44.944104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:44.944104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:44.944104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:44.944104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:44.944104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:44.944104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:44.944104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:44.944104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:44.944104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:44.944104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:44.944104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:44.945889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:33:44.945889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:33:44.945889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:33:44.945889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:33:44.945889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:33:44.945889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:33:44.945889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:44.945889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:44.945889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 406/798] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:33:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:44.945889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a0907-1b15-4650-bf00-519076850a3e None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:44.947996 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a0907-1b15-4650-bf00-519076850a3e None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:44.947996 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a0907-1b15-4650-bf00-519076850a3e None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:44.951807 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 407/799] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:33:44.952834 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6a0907-1b15-4650-bf00-519076850a3e None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:44.988385 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a73ba7-631c-4b2d-aaf7-d1085928a447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:44.989036 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a73ba7-631c-4b2d-aaf7-d1085928a447 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:44.989829 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a73ba7-631c-4b2d-aaf7-d1085928a447 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:44.990243 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a73ba7-631c-4b2d-aaf7-d1085928a447 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:44.998623 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a73ba7-631c-4b2d-aaf7-d1085928a447 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:45.000174 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a73ba7-631c-4b2d-aaf7-d1085928a447 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:45.014845 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-25a73ba7-631c-4b2d-aaf7-d1085928a447 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 02:33:45.014845 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:45.014845 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:45.014845 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:45.014845 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:45.014845 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:45.014845 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:45.014845 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:45.014845 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:45.014845 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:45.014845 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:45.014845 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:45.014845 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:33:45.014845 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:33:45.014845 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:33:45.014845 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:33:45.016703 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:45.016703 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:45.016703 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:33:45.016703 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:45.016703 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:33:45.016703 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:45.016703 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:45.016703 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:45.016703 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:45.016703 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:45.016703 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:45.016703 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:45.016703 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:45.016703 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:45.016703 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:45.016703 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:45.016703 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:45.016703 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:45.019846 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:45.019846 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:45.019846 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:33:45.019846 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:33:45.019846 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:33:45.019846 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:33:45.019846 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:33:45.019846 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:33:45.019846 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 02:33:45.019846 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:45.019846 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 393/800] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Sep 22 02:33:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 433 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:33:45.019846 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9896c-5a5d-48ea-a21d-07b12993de19 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:45.019846 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9896c-5a5d-48ea-a21d-07b12993de19 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:45.022928 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9896c-5a5d-48ea-a21d-07b12993de19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:45.044931 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-e0b9896c-5a5d-48ea-a21d-07b12993de19 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:45.048209 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-e0b9896c-5a5d-48ea-a21d-07b12993de19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:45.132993 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6a0907-1b15-4650-bf00-519076850a3e None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:45.162395 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:45.175590 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b92f1-aa5d-460e-b425-9f5f3277b5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:45.176617 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b92f1-aa5d-460e-b425-9f5f3277b5f7 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:45.177640 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b92f1-aa5d-460e-b425-9f5f3277b5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:45.178109 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b92f1-aa5d-460e-b425-9f5f3277b5f7 None admin] PATH_INFO: `/v3/projects` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:45.179047 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b92f1-aa5d-460e-b425-9f5f3277b5f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:45.179919 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b92f1-aa5d-460e-b425-9f5f3277b5f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:45.182648 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b92f1-aa5d-460e-b425-9f5f3277b5f7 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:45.197765 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 408/801] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Sep 22 02:33:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:45.198990 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b92f1-aa5d-460e-b425-9f5f3277b5f7 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:45.218850 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 394/802] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:33:45.218850 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9896c-5a5d-48ea-a21d-07b12993de19 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:45.270406 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:45.417071 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b92f1-aa5d-460e-b425-9f5f3277b5f7 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:45.433075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9896c-5a5d-48ea-a21d-07b12993de19 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:45.455895 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dece33d-745f-46f0-992d-544f2105b2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:45.456411 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dece33d-745f-46f0-992d-544f2105b2ec None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:45.456711 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dece33d-745f-46f0-992d-544f2105b2ec None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:45.456970 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dece33d-745f-46f0-992d-544f2105b2ec None admin] PATH_INFO: `/v3/users` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:45.457702 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dece33d-745f-46f0-992d-544f2105b2ec None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:45.458039 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dece33d-745f-46f0-992d-544f2105b2ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:45.459514 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dece33d-745f-46f0-992d-544f2105b2ec None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:45.489051 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6363127-ed39-41bb-a928-508b601a0c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:45.490401 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6363127-ed39-41bb-a928-508b601a0c74 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:45.490749 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6363127-ed39-41bb-a928-508b601a0c74 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:45.492293 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6363127-ed39-41bb-a928-508b601a0c74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:45.495335 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6363127-ed39-41bb-a928-508b601a0c74 None admin] RBAC: Adding query filter params (role.id=31a687548f55493b8ea666fe7e350303, scope.project.id=2a792f8dc45145d687ddd58c94d44b5e, user.id=a64ed6c8e15d479887ab1dc59997d715) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:45.495664 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6363127-ed39-41bb-a928-508b601a0c74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:45.498046 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6363127-ed39-41bb-a928-508b601a0c74 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:45.499443 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 409/803] 10.222.0.20 () {56 vars in 1149 bytes} [Thu Sep 22 02:33:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:45.501961 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7dc2fe-8638-405b-8453-96deef0ee5e4 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:45.502452 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7dc2fe-8638-405b-8453-96deef0ee5e4 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:45.502954 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7dc2fe-8638-405b-8453-96deef0ee5e4 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:45.505661 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 410/804] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:33:45.507847 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7dc2fe-8638-405b-8453-96deef0ee5e4 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:45.513297 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 395/805] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:33:45 2022] GET /identity/v3/role_assignments?user.id=a64ed6c8e15d479887ab1dc59997d715&scope.project.id=2a792f8dc45145d687ddd58c94d44b5e&role.id=31a687548f55493b8ea666fe7e350303 => generated 608 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:45.514671 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e50224-d8db-40b4-9d6a-0d72fb999496 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:45.514994 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e50224-d8db-40b4-9d6a-0d72fb999496 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:45.515269 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e50224-d8db-40b4-9d6a-0d72fb999496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:45.560876 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-f7e50224-d8db-40b4-9d6a-0d72fb999496 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:45.570862 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-f7e50224-d8db-40b4-9d6a-0d72fb999496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:45.613406 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:45.788255 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7dc2fe-8638-405b-8453-96deef0ee5e4 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:45.809152 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 396/806] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:33:45.810261 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e50224-d8db-40b4-9d6a-0d72fb999496 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:45.839550 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:45.868644 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e814d33-21fd-4c07-8dab-8b5fbf9e033b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:45.869310 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e814d33-21fd-4c07-8dab-8b5fbf9e033b None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:45.869657 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e814d33-21fd-4c07-8dab-8b5fbf9e033b None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:45.869966 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e814d33-21fd-4c07-8dab-8b5fbf9e033b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:45.893491 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e814d33-21fd-4c07-8dab-8b5fbf9e033b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:45.898312 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e814d33-21fd-4c07-8dab-8b5fbf9e033b None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:45.934018 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-4e814d33-21fd-4c07-8dab-8b5fbf9e033b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:45.934018 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:45.934018 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:45.934018 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:45.934018 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:45.934018 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:45.934018 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:45.934018 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:45.934018 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:45.934018 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:45.934018 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:45.934018 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:45.934018 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:45.934018 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:45.934018 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:45.934018 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:45.935769 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:45.935769 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:45.935769 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:45.935769 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:45.935769 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:45.935769 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:45.935769 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:45.935769 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:45.935769 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:45.935769 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:45.935769 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:45.935769 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:45.935769 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:45.935769 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:45.935769 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:45.935769 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:45.935769 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:45.938456 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:45.938456 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:45.938456 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:45.938456 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 411/807] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 431 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:45.943800 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e814d33-21fd-4c07-8dab-8b5fbf9e033b None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:46.026769 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e50224-d8db-40b4-9d6a-0d72fb999496 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:46.098537 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790dc885-ddf2-4988-95fb-5371ac691787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:46.099328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790dc885-ddf2-4988-95fb-5371ac691787 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:46.099623 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790dc885-ddf2-4988-95fb-5371ac691787 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:46.099905 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790dc885-ddf2-4988-95fb-5371ac691787 None admin] PATH_INFO: `/v3/services/network` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:46.100515 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790dc885-ddf2-4988-95fb-5371ac691787 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:46.101939 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790dc885-ddf2-4988-95fb-5371ac691787 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:46.118310 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-790dc885-ddf2-4988-95fb-5371ac691787 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 02:33:46.118310 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:46.118310 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 02:33:46.118310 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 02:33:46.118310 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 02:33:46.118310 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 02:33:46.118310 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 02:33:46.118310 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:33:46.118310 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 02:33:46.118310 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:46.118310 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 02:33:46.118310 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:46.118310 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:46.118310 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:46.118310 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:46.118310 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:46.118310 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:46.120732 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:46.120732 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:46.120732 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:46.120732 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:46.120732 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:46.120732 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:46.120732 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 02:33:46.120732 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 02:33:46.120732 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 02:33:46.120732 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 02:33:46.120732 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:46.120732 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:46.120732 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:46.120732 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:46.120732 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:46.120732 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:46.122378 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:46.122378 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:46.122378 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:46.122378 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:46.122378 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:46.122378 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:46.122378 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:46.122378 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:46.122378 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:46.122378 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:46.122378 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 02:33:46.122378 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:33:46.122378 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 02:33:46.122378 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:46.122378 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 397/808] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:33:45 2022] GET /identity/v3/services/network => generated 88 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:46.131934 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790dc885-ddf2-4988-95fb-5371ac691787 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:46.172137 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:46.192149 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e814d33-21fd-4c07-8dab-8b5fbf9e033b None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:46.268524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d4d33-3ad6-4a59-ac1f-066621ccb33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:46.269157 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d4d33-3ad6-4a59-ac1f-066621ccb33b None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:46.269439 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d4d33-3ad6-4a59-ac1f-066621ccb33b None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:46.271072 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d4d33-3ad6-4a59-ac1f-066621ccb33b None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:46.271839 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023d4d33-3ad6-4a59-ac1f-066621ccb33b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:46.272156 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023d4d33-3ad6-4a59-ac1f-066621ccb33b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:46.273839 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023d4d33-3ad6-4a59-ac1f-066621ccb33b None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:46.306250 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 412/809] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:46.313166 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d4d33-3ad6-4a59-ac1f-066621ccb33b None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:46.320046 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:46.360316 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790dc885-ddf2-4988-95fb-5371ac691787 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:46.433223 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431d43a-bb5a-4ea7-97b0-5e6629ddef76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:46.434756 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431d43a-bb5a-4ea7-97b0-5e6629ddef76 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:46.435565 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431d43a-bb5a-4ea7-97b0-5e6629ddef76 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:46.436464 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431d43a-bb5a-4ea7-97b0-5e6629ddef76 None admin] PATH_INFO: `/v3/services` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:46.437875 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1431d43a-bb5a-4ea7-97b0-5e6629ddef76 None admin] RBAC: Adding query filter params (name=network) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:46.438697 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1431d43a-bb5a-4ea7-97b0-5e6629ddef76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:46.441934 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1431d43a-bb5a-4ea7-97b0-5e6629ddef76 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:46.450534 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 398/810] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Sep 22 02:33:46 2022] GET /identity/v3/services?name=network => generated 124 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:46.452479 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431d43a-bb5a-4ea7-97b0-5e6629ddef76 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:46.539954 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d4d33-3ad6-4a59-ac1f-066621ccb33b None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:46.616952 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f4f061-784e-48fa-b399-381bebac0218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:46.618441 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4f061-784e-48fa-b399-381bebac0218 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:46.619121 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4f061-784e-48fa-b399-381bebac0218 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:46.619786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4f061-784e-48fa-b399-381bebac0218 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:46.632969 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431d43a-bb5a-4ea7-97b0-5e6629ddef76 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:46.635146 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f4f061-784e-48fa-b399-381bebac0218 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:46.639417 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f4f061-784e-48fa-b399-381bebac0218 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:46.663626 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-13f4f061-784e-48fa-b399-381bebac0218 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:46.663626 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:46.663626 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:46.663626 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:46.663626 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:46.663626 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:46.663626 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:46.663626 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:46.663626 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:46.663626 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:46.663626 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:46.663626 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:46.663626 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:33:46.663626 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:33:46.663626 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:33:46.665465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:33:46.665465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:46.665465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:46.665465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:46.665465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:46.665465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:46.665465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:46.665465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:46.665465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:46.665465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:46.665465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:46.665465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:46.665465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:46.665465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:46.665465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:46.665465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:46.665465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:46.667369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:33:46.667369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:33:46.667369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:33:46.667369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:33:46.667369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:33:46.667369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:33:46.667369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:46.667369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:46.671547 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 413/811] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:33:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:46.673218 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f4f061-784e-48fa-b399-381bebac0218 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:46.699502 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb90a4-fce7-49c5-98f1-0119b0c7a635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:46.700662 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb90a4-fce7-49c5-98f1-0119b0c7a635 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:46.701265 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb90a4-fce7-49c5-98f1-0119b0c7a635 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:46.701888 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb90a4-fce7-49c5-98f1-0119b0c7a635 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:46.706849 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:46.721772 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb90a4-fce7-49c5-98f1-0119b0c7a635 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:33:46.731839 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-62fb90a4-fce7-49c5-98f1-0119b0c7a635 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:33:46.731839 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:46.731839 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:46.731839 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:46.731839 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:46.731839 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:46.731839 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:46.731839 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:46.731839 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:46.731839 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:46.731839 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:46.731839 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:46.731839 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:33:46.731839 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:33:46.731839 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:33:46.731839 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:33:46.733884 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:33:46.733884 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:33:46.733884 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:33:46.733884 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:33:46.733884 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:46.733884 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:46.733884 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:46.733884 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:46.733884 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:46.733884 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:46.733884 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:46.733884 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:46.733884 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:46.733884 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:46.733884 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:46.733884 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:46.736064 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:46.736064 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:46.736064 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:46.736064 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:46.736064 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:33:46.736064 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:33:46.736064 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:33:46.736064 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:33:46.736064 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:33:46.736064 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:33:46.736064 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:33:46.736064 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:46.738185 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 399/812] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Sep 22 02:33:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:46.740065 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb90a4-fce7-49c5-98f1-0119b0c7a635 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:46.755900 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:46.901570 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f4f061-784e-48fa-b399-381bebac0218 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:46.921836 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb90a4-fce7-49c5-98f1-0119b0c7a635 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:46.949955 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d828e4-11f2-4eb9-82d6-804bf825d0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:46.950592 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d828e4-11f2-4eb9-82d6-804bf825d0f0 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:46.950906 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d828e4-11f2-4eb9-82d6-804bf825d0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:46.951188 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d828e4-11f2-4eb9-82d6-804bf825d0f0 None admin] PATH_INFO: `/v3/services` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:46.951861 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d828e4-11f2-4eb9-82d6-804bf825d0f0 None admin] RBAC: Adding query filter params (type=network) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:46.952149 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d828e4-11f2-4eb9-82d6-804bf825d0f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:46.953551 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d828e4-11f2-4eb9-82d6-804bf825d0f0 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:46.963535 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 414/813] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Sep 22 02:33:46 2022] GET /identity/v3/services?type=network => generated 350 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:46.963535 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d828e4-11f2-4eb9-82d6-804bf825d0f0 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:46.965900 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e31cd-a567-4d2c-b59a-9d6a7e48f91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:46.966177 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e31cd-a567-4d2c-b59a-9d6a7e48f91b None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:46.966459 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e31cd-a567-4d2c-b59a-9d6a7e48f91b None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:46.966732 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e31cd-a567-4d2c-b59a-9d6a7e48f91b None admin] PATH_INFO: `/v3/projects` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:46.967501 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e31cd-a567-4d2c-b59a-9d6a7e48f91b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:46.967896 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e31cd-a567-4d2c-b59a-9d6a7e48f91b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:46.969285 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e31cd-a567-4d2c-b59a-9d6a7e48f91b None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:46.987767 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 400/814] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Sep 22 02:33:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:46.987767 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61797424-10ee-41f8-b43d-b64a6c892506 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:46.987767 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61797424-10ee-41f8-b43d-b64a6c892506 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:46.987767 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61797424-10ee-41f8-b43d-b64a6c892506 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:46.987767 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 401/815] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:33:46.987767 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61797424-10ee-41f8-b43d-b64a6c892506 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:47.069198 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:47.108966 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:47.120971 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61797424-10ee-41f8-b43d-b64a6c892506 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:47.148193 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d828e4-11f2-4eb9-82d6-804bf825d0f0 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:47.163247 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a0e17-e689-4891-b610-e3e2e4d6ebed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:47.164141 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a0e17-e689-4891-b610-e3e2e4d6ebed None admin] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:47.164141 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a0e17-e689-4891-b610-e3e2e4d6ebed None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:47.164333 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a0e17-e689-4891-b610-e3e2e4d6ebed None admin] PATH_INFO: `/v3/endpoints` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:47.165061 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1a0e17-e689-4891-b610-e3e2e4d6ebed None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:47.166516 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1a0e17-e689-4891-b610-e3e2e4d6ebed None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:47.202856 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 402/816] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Sep 22 02:33:46 2022] POST /identity/v3/endpoints => generated 337 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:33:47.202856 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a0e17-e689-4891-b610-e3e2e4d6ebed None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:47.227254 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c1878f-4c7e-460f-9255-9b46922c59e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:47.228253 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1878f-4c7e-460f-9255-9b46922c59e2 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:47.228728 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1878f-4c7e-460f-9255-9b46922c59e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:47.229178 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1878f-4c7e-460f-9255-9b46922c59e2 None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:47.230395 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c1878f-4c7e-460f-9255-9b46922c59e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:47.230950 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c1878f-4c7e-460f-9255-9b46922c59e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:47.233511 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c1878f-4c7e-460f-9255-9b46922c59e2 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:47.250181 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 415/817] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:33:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:47.252559 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc0463b-571b-49e3-b7e8-c5a4a14c98ec None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:47.253019 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc0463b-571b-49e3-b7e8-c5a4a14c98ec None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:47.253473 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc0463b-571b-49e3-b7e8-c5a4a14c98ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:47.285792 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-3fc0463b-571b-49e3-b7e8-c5a4a14c98ec None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:47.287969 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-3fc0463b-571b-49e3-b7e8-c5a4a14c98ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:47.423841 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a0e17-e689-4891-b610-e3e2e4d6ebed None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:47.433781 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 416/818] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:47 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:33:47.435238 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc0463b-571b-49e3-b7e8-c5a4a14c98ec None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:47.461623 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:47.490137 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d35351-c888-401b-9371-5ece4c849d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:47.491242 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d35351-c888-401b-9371-5ece4c849d8a None admin] REQUEST_METHOD: `PUT` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:47.491771 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d35351-c888-401b-9371-5ece4c849d8a None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:47.492243 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d35351-c888-401b-9371-5ece4c849d8a None admin] PATH_INFO: `/v3/projects/2a792f8dc45145d687ddd58c94d44b5e/users/9821de2a961f4ddab2fca8e45b74e3db/roles/fb2fbee669e14ffbbe0aa70642eab1ce` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:47.518222 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d35351-c888-401b-9371-5ece4c849d8a None admin] RBAC: Authorizing `identity:create_grant(project_id=2a792f8dc45145d687ddd58c94d44b5e, user_id=9821de2a961f4ddab2fca8e45b74e3db, role_id=fb2fbee669e14ffbbe0aa70642eab1ce)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:47.521006 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d35351-c888-401b-9371-5ece4c849d8a None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:47.544856 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 403/819] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Sep 22 02:33:47 2022] PUT /identity/v3/projects/2a792f8dc45145d687ddd58c94d44b5e/users/9821de2a961f4ddab2fca8e45b74e3db/roles/fb2fbee669e14ffbbe0aa70642eab1ce => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 02:33:47.560410 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:47.610001 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc0463b-571b-49e3-b7e8-c5a4a14c98ec None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:47.671462 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a18175-80d7-4ef9-be62-b874c7f5d4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:47.672619 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a18175-80d7-4ef9-be62-b874c7f5d4e9 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:47.673207 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a18175-80d7-4ef9-be62-b874c7f5d4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:47.673654 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a18175-80d7-4ef9-be62-b874c7f5d4e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:47.690471 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a18175-80d7-4ef9-be62-b874c7f5d4e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:47.692740 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a18175-80d7-4ef9-be62-b874c7f5d4e9 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:47.716516 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-e4a18175-80d7-4ef9-be62-b874c7f5d4e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:47.716516 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:47.716516 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:47.716516 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:47.716516 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:47.716516 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:47.716516 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:47.716516 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:47.716516 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:47.716516 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:47.716516 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:47.716516 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:47.716516 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:47.716516 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:47.716516 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:47.716516 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:47.718383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:47.718383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:47.718383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:47.718383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:47.718383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:47.718383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:47.718383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:47.718383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:47.718383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:47.718383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:47.718383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:47.718383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:47.718383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:47.718383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:47.718383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:47.718383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:47.718383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:47.719864 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:47.719864 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:47.719864 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:47.722339 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 417/820] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:47.730077 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d35351-c888-401b-9371-5ece4c849d8a None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:47.792549 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a18175-80d7-4ef9-be62-b874c7f5d4e9 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:47.892398 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d35351-c888-401b-9371-5ece4c849d8a None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:47.951356 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99dcdb9-a2be-4129-96fb-daddcbfea93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:47.952267 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99dcdb9-a2be-4129-96fb-daddcbfea93e None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:47.952843 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99dcdb9-a2be-4129-96fb-daddcbfea93e None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:47.953307 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99dcdb9-a2be-4129-96fb-daddcbfea93e None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:47.954410 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99dcdb9-a2be-4129-96fb-daddcbfea93e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:47.954953 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99dcdb9-a2be-4129-96fb-daddcbfea93e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:47.957288 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99dcdb9-a2be-4129-96fb-daddcbfea93e None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:47.976169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a18175-80d7-4ef9-be62-b874c7f5d4e9 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:47.978339 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 404/821] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:47.984190 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99dcdb9-a2be-4129-96fb-daddcbfea93e None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:48.008067 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:48.042524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76b260b-1656-4cbd-b3d1-da2c21a11044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:48.043665 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b260b-1656-4cbd-b3d1-da2c21a11044 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:48.044166 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b260b-1656-4cbd-b3d1-da2c21a11044 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:48.044641 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b260b-1656-4cbd-b3d1-da2c21a11044 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:48.045445 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76b260b-1656-4cbd-b3d1-da2c21a11044 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:48.047079 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:48.049072 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76b260b-1656-4cbd-b3d1-da2c21a11044 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:48.061520 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-a76b260b-1656-4cbd-b3d1-da2c21a11044 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 02:33:48.061520 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:48.061520 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 02:33:48.061520 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 02:33:48.061520 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 02:33:48.061520 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 02:33:48.061520 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 02:33:48.061520 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:33:48.061520 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 02:33:48.061520 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:48.061520 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 02:33:48.061520 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:48.061520 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:48.061520 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:48.061520 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:48.061520 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:48.061520 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:48.062601 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:48.062601 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:48.062601 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:48.062601 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:48.062601 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:48.062601 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:48.062601 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 02:33:48.062601 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 02:33:48.062601 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 02:33:48.062601 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 02:33:48.062601 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:48.062601 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:48.062601 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:48.062601 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:48.062601 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:48.062601 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:48.063484 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:48.063484 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:48.063484 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:48.063484 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:48.063484 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:48.063484 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:48.063484 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:48.063484 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:48.063484 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:48.063484 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:48.063484 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 02:33:48.063484 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:33:48.063484 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 02:33:48.063484 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:48.068080 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 418/822] 10.222.0.20 () {56 vars in 1129 bytes} [Thu Sep 22 02:33:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:48.079052 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76b260b-1656-4cbd-b3d1-da2c21a11044 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:48.171190 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99dcdb9-a2be-4129-96fb-daddcbfea93e None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:48.245188 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76b260b-1656-4cbd-b3d1-da2c21a11044 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:48.246167 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca526f4-5fe7-4c8d-a001-7ba5a2e312e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:48.247083 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca526f4-5fe7-4c8d-a001-7ba5a2e312e5 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:48.247776 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca526f4-5fe7-4c8d-a001-7ba5a2e312e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:48.248242 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca526f4-5fe7-4c8d-a001-7ba5a2e312e5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:48.259729 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca526f4-5fe7-4c8d-a001-7ba5a2e312e5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:48.261865 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca526f4-5fe7-4c8d-a001-7ba5a2e312e5 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:48.287318 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-bca526f4-5fe7-4c8d-a001-7ba5a2e312e5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 22 02:33:48.287318 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:48.287318 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:48.287318 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:48.287318 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:48.287318 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:48.287318 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:48.287318 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:48.287318 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:48.287318 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:48.287318 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:48.287318 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:48.287318 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:33:48.287318 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:33:48.287318 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:33:48.287318 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:33:48.289113 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:48.289113 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:48.289113 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:33:48.289113 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:48.289113 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:33:48.289113 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:48.289113 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:48.289113 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:48.289113 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:48.289113 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:48.289113 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:48.289113 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:48.289113 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:48.289113 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:48.289113 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:48.289113 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:48.289113 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:48.289113 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:48.290248 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:48.290248 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:48.290248 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:33:48.290248 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:33:48.290248 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:33:48.290248 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:33:48.290248 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:33:48.290248 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:33:48.290248 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 22 02:33:48.290248 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:48.290248 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 405/823] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Sep 22 02:33:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:48.295199 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca526f4-5fe7-4c8d-a001-7ba5a2e312e5 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:48.307323 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3686bf-5766-4969-a30b-64dfb67c4b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:48.308636 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3686bf-5766-4969-a30b-64dfb67c4b2a None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:48.309284 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3686bf-5766-4969-a30b-64dfb67c4b2a None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:48.309915 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3686bf-5766-4969-a30b-64dfb67c4b2a None admin] PATH_INFO: `/v3/services` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:48.312080 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3686bf-5766-4969-a30b-64dfb67c4b2a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:48.312821 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3686bf-5766-4969-a30b-64dfb67c4b2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:48.315611 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3686bf-5766-4969-a30b-64dfb67c4b2a None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:48.332682 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 419/824] 10.222.0.20 () {56 vars in 1124 bytes} [Thu Sep 22 02:33:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:48.336069 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3686bf-5766-4969-a30b-64dfb67c4b2a None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:48.427272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:48.476147 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:48.495627 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca526f4-5fe7-4c8d-a001-7ba5a2e312e5 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:48.527350 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3686bf-5766-4969-a30b-64dfb67c4b2a None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:48.562390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f8d9b-393d-4d16-a219-4324956f4ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:48.563644 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f8d9b-393d-4d16-a219-4324956f4ea6 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:48.564252 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f8d9b-393d-4d16-a219-4324956f4ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:48.564861 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f8d9b-393d-4d16-a219-4324956f4ea6 None admin] PATH_INFO: `/v3/users` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:48.566771 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f8d9b-393d-4d16-a219-4324956f4ea6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:48.566959 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f8d9b-393d-4d16-a219-4324956f4ea6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:48.570408 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f8d9b-393d-4d16-a219-4324956f4ea6 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:48.578655 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1db3b-fc02-4413-a8ea-d5255f6ebe8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:48.579585 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1db3b-fc02-4413-a8ea-d5255f6ebe8b None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:48.580053 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1db3b-fc02-4413-a8ea-d5255f6ebe8b None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:48.580494 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1db3b-fc02-4413-a8ea-d5255f6ebe8b None admin] PATH_INFO: `/v3/services` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:48.581324 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1db3b-fc02-4413-a8ea-d5255f6ebe8b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:48.581801 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1db3b-fc02-4413-a8ea-d5255f6ebe8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:48.589185 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1db3b-fc02-4413-a8ea-d5255f6ebe8b None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:48.603644 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 420/825] 10.222.0.20 () {56 vars in 1124 bytes} [Thu Sep 22 02:33:48 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:33:48.607565 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1db3b-fc02-4413-a8ea-d5255f6ebe8b None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:48.610208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 406/826] 10.222.0.20 () {56 vars in 1135 bytes} [Thu Sep 22 02:33:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:48.616095 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f8d9b-393d-4d16-a219-4324956f4ea6 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:48.736637 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f8d9b-393d-4d16-a219-4324956f4ea6 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:48.779179 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2952ad-f144-4a61-97d3-8f70c844b9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:48.779751 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2952ad-f144-4a61-97d3-8f70c844b9bd None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:48.780030 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2952ad-f144-4a61-97d3-8f70c844b9bd None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:48.780313 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2952ad-f144-4a61-97d3-8f70c844b9bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:48.784905 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1db3b-fc02-4413-a8ea-d5255f6ebe8b None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:48.792826 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2952ad-f144-4a61-97d3-8f70c844b9bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:48.794298 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2952ad-f144-4a61-97d3-8f70c844b9bd None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:48.797822 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:48.813470 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:48.831853 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-6c2952ad-f144-4a61-97d3-8f70c844b9bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:48.831853 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:48.831853 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:48.831853 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:48.831853 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:48.831853 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:48.831853 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:48.831853 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:48.831853 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:48.831853 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:48.831853 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:48.831853 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:48.831853 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:48.831853 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:48.831853 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:48.831853 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:48.834046 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:48.834046 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:48.834046 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:48.834046 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:48.834046 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:48.834046 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:48.834046 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:48.834046 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:48.834046 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:48.834046 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:48.834046 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:48.834046 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:48.834046 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:48.834046 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:48.834046 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:48.834046 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:48.834046 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:48.835932 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:48.835932 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:48.835932 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:48.836336 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51152c0c-5a8d-47d4-973d-fae058d16417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:48.836336 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51152c0c-5a8d-47d4-973d-fae058d16417 None admin] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:48.836336 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51152c0c-5a8d-47d4-973d-fae058d16417 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:48.836723 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51152c0c-5a8d-47d4-973d-fae058d16417 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:48.837940 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51152c0c-5a8d-47d4-973d-fae058d16417 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:48.839473 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 407/827] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:33:48.840781 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22f1d8-3454-49c1-948a-912ad840935f None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:48.841048 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22f1d8-3454-49c1-948a-912ad840935f None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:48.841329 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22f1d8-3454-49c1-948a-912ad840935f None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:48.842917 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 408/828] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:33:48.847809 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec22f1d8-3454-49c1-948a-912ad840935f None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:48.848097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51152c0c-5a8d-47d4-973d-fae058d16417 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:48.898860 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 421/829] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Sep 22 02:33:48 2022] POST /identity/v3/endpoints => generated 346 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:33:48.898860 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7cf9b-ae74-475c-a7ca-b9256d75810e None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:48.898860 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7cf9b-ae74-475c-a7ca-b9256d75810e None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:48.898860 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7cf9b-ae74-475c-a7ca-b9256d75810e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:48.928133 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-beb7cf9b-ae74-475c-a7ca-b9256d75810e None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:48.929915 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-beb7cf9b-ae74-475c-a7ca-b9256d75810e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:49.043474 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec22f1d8-3454-49c1-948a-912ad840935f None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:49.077959 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 422/830] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:48 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:33:49.088825 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938572b2-a1e2-4081-a61d-6468a1e545e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:49.089471 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938572b2-a1e2-4081-a61d-6468a1e545e9 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:49.089758 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938572b2-a1e2-4081-a61d-6468a1e545e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:49.090184 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938572b2-a1e2-4081-a61d-6468a1e545e9 None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:49.090724 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938572b2-a1e2-4081-a61d-6468a1e545e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:49.091070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938572b2-a1e2-4081-a61d-6468a1e545e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:49.092515 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938572b2-a1e2-4081-a61d-6468a1e545e9 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:49.106392 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 409/831] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:49.115566 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7cf9b-ae74-475c-a7ca-b9256d75810e None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:49.240454 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:49.248322 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7cf9b-ae74-475c-a7ca-b9256d75810e None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:49.312140 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf4d237-b11e-4691-bdb3-33e75b1969f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:49.313597 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf4d237-b11e-4691-bdb3-33e75b1969f4 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:49.314308 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf4d237-b11e-4691-bdb3-33e75b1969f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:49.315061 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf4d237-b11e-4691-bdb3-33e75b1969f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:49.328221 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf4d237-b11e-4691-bdb3-33e75b1969f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:49.331288 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf4d237-b11e-4691-bdb3-33e75b1969f4 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:49.355818 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-ebf4d237-b11e-4691-bdb3-33e75b1969f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:49.355818 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:49.355818 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:49.355818 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:49.355818 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:49.355818 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:49.355818 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:49.355818 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:49.355818 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:49.355818 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:49.355818 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:49.355818 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:49.355818 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:33:49.355818 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:33:49.355818 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:33:49.357881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:33:49.357881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:49.357881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:49.357881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:49.357881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:49.357881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:49.357881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:49.357881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:49.357881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:49.357881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:49.357881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:49.357881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:49.357881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:49.357881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:49.357881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:49.357881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:49.357881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:49.360003 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:33:49.360003 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:33:49.360003 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:33:49.360003 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:33:49.360003 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:33:49.360003 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:33:49.360003 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:49.360003 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:49.364120 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 423/832] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:33:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:49.367957 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938572b2-a1e2-4081-a61d-6468a1e545e9 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:49.371979 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf4d237-b11e-4691-bdb3-33e75b1969f4 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:49.445158 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:49.506388 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938572b2-a1e2-4081-a61d-6468a1e545e9 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:49.535417 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf4d237-b11e-4691-bdb3-33e75b1969f4 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:49.555882 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428495b-0035-4b2f-8fe7-b04e2daf8d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:49.556959 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428495b-0035-4b2f-8fe7-b04e2daf8d6a None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:49.557449 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428495b-0035-4b2f-8fe7-b04e2daf8d6a None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:49.557936 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428495b-0035-4b2f-8fe7-b04e2daf8d6a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:49.559133 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428495b-0035-4b2f-8fe7-b04e2daf8d6a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:49.561120 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428495b-0035-4b2f-8fe7-b04e2daf8d6a None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:49.574071 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-8428495b-0035-4b2f-8fe7-b04e2daf8d6a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 02:33:49.574071 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:49.574071 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 02:33:49.574071 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 02:33:49.574071 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 02:33:49.574071 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 02:33:49.574071 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 02:33:49.574071 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:33:49.574071 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 02:33:49.574071 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:49.574071 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 02:33:49.574071 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:49.574071 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:49.574071 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:49.574071 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:49.574071 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:49.576531 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:49.576531 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:49.576531 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:49.576531 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:49.576531 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:49.576531 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:49.576531 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:49.576531 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 02:33:49.576531 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 02:33:49.576531 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 02:33:49.576531 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 02:33:49.576531 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:49.576531 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:49.576531 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:49.576531 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:49.576531 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:49.576531 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:49.578300 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:49.578300 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:49.578300 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:49.578300 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:49.578300 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:49.578300 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:49.578300 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:49.578300 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:49.578300 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:49.578300 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:49.578300 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 02:33:49.578300 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:33:49.578300 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 02:33:49.578300 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:49.578300 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 410/833] 10.222.0.20 () {56 vars in 1132 bytes} [Thu Sep 22 02:33:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:49.583564 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428495b-0035-4b2f-8fe7-b04e2daf8d6a None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:49.612139 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf61752-533d-45ad-99e3-af0f21d33707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:49.613630 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf61752-533d-45ad-99e3-af0f21d33707 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:49.614688 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf61752-533d-45ad-99e3-af0f21d33707 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:49.615542 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf61752-533d-45ad-99e3-af0f21d33707 None admin] PATH_INFO: `/v3/projects` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:49.617747 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf61752-533d-45ad-99e3-af0f21d33707 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:49.618716 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf61752-533d-45ad-99e3-af0f21d33707 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:49.622134 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf61752-533d-45ad-99e3-af0f21d33707 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:49.645114 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 424/834] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Sep 22 02:33:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:49.647268 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf61752-533d-45ad-99e3-af0f21d33707 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:49.669138 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:49.733435 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428495b-0035-4b2f-8fe7-b04e2daf8d6a None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:49.760663 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:49.766773 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf61752-533d-45ad-99e3-af0f21d33707 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:49.778001 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4898b06-7192-4655-85bd-dd1546ff8b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:49.778787 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4898b06-7192-4655-85bd-dd1546ff8b9c None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:49.779213 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4898b06-7192-4655-85bd-dd1546ff8b9c None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:49.779619 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4898b06-7192-4655-85bd-dd1546ff8b9c None admin] PATH_INFO: `/v3/services` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:49.780554 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4898b06-7192-4655-85bd-dd1546ff8b9c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:49.781021 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4898b06-7192-4655-85bd-dd1546ff8b9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:49.782640 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4898b06-7192-4655-85bd-dd1546ff8b9c None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:49.793800 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 411/835] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Sep 22 02:33:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:49.794540 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4898b06-7192-4655-85bd-dd1546ff8b9c None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:49.810117 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea72e283-2d4f-47c2-a1cf-d8461050dcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:49.811209 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72e283-2d4f-47c2-a1cf-d8461050dcb4 None admin] REQUEST_METHOD: `PUT` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:49.811632 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72e283-2d4f-47c2-a1cf-d8461050dcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:49.812079 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72e283-2d4f-47c2-a1cf-d8461050dcb4 None admin] PATH_INFO: `/v3/projects/2a792f8dc45145d687ddd58c94d44b5e/users/4b9ad548622b4e54a6fecd0afd5ef3c2/roles/31a687548f55493b8ea666fe7e350303` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:49.814403 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea72e283-2d4f-47c2-a1cf-d8461050dcb4 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a792f8dc45145d687ddd58c94d44b5e, user_id=4b9ad548622b4e54a6fecd0afd5ef3c2, role_id=31a687548f55493b8ea666fe7e350303)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:49.816335 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea72e283-2d4f-47c2-a1cf-d8461050dcb4 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:49.846889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 425/836] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Sep 22 02:33:49 2022] PUT /identity/v3/projects/2a792f8dc45145d687ddd58c94d44b5e/users/4b9ad548622b4e54a6fecd0afd5ef3c2/roles/31a687548f55493b8ea666fe7e350303 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 02:33:50.009666 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4898b06-7192-4655-85bd-dd1546ff8b9c None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:50.090951 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f4b0c4-77b6-4de6-a84f-0bd4876d752c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:50.091591 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4b0c4-77b6-4de6-a84f-0bd4876d752c None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:50.091875 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4b0c4-77b6-4de6-a84f-0bd4876d752c None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:50.092152 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4b0c4-77b6-4de6-a84f-0bd4876d752c None admin] PATH_INFO: `/v3/services` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:50.092811 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f4b0c4-77b6-4de6-a84f-0bd4876d752c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:50.093122 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f4b0c4-77b6-4de6-a84f-0bd4876d752c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:50.094525 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f4b0c4-77b6-4de6-a84f-0bd4876d752c None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:50.100581 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 412/837] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Sep 22 02:33:49 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:50.103846 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3962ed8-2bde-4c3f-a18c-3b666ebb6330 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:50.104154 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3962ed8-2bde-4c3f-a18c-3b666ebb6330 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:50.104417 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3962ed8-2bde-4c3f-a18c-3b666ebb6330 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:50.106203 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 426/838] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:33:50.112292 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76992a-f7f6-4e16-9796-41c8c1d45f8a None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:50.112637 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76992a-f7f6-4e16-9796-41c8c1d45f8a None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:50.112862 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76992a-f7f6-4e16-9796-41c8c1d45f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:50.134625 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-9e76992a-f7f6-4e16-9796-41c8c1d45f8a None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:50.135920 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-9e76992a-f7f6-4e16-9796-41c8c1d45f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:50.196093 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:50.228135 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 413/839] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:50 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:33:50.552129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3962ed8-2bde-4c3f-a18c-3b666ebb6330 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:50.696873 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:50.747634 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3962ed8-2bde-4c3f-a18c-3b666ebb6330 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:50.818148 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ba2f1-977c-4269-98a7-e5cfcc8a84cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:50.819297 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ba2f1-977c-4269-98a7-e5cfcc8a84cf None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:50.819773 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ba2f1-977c-4269-98a7-e5cfcc8a84cf None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:50.820250 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ba2f1-977c-4269-98a7-e5cfcc8a84cf None admin] PATH_INFO: `/v3/roles/service` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:50.840874 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5ba2f1-977c-4269-98a7-e5cfcc8a84cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:33:50.851228 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-dd5ba2f1-977c-4269-98a7-e5cfcc8a84cf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:33:50.851228 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:50.851228 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:50.851228 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:50.851228 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:50.851228 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:50.851228 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:50.851228 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:50.851228 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:50.851228 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:50.851228 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:50.851228 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:50.851228 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:33:50.851228 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:33:50.851228 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:33:50.851228 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:33:50.852254 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:33:50.852254 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:33:50.852254 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:33:50.852254 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:33:50.852254 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:50.852254 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:50.852254 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:50.852254 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:50.852254 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:50.852254 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:50.852254 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:50.852254 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:50.852254 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:50.852254 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:50.852254 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:50.852254 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:50.853114 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:50.853114 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:50.853114 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:50.853114 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:50.853114 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:33:50.853114 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:33:50.853114 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:33:50.853114 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:33:50.853114 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:33:50.853114 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:33:50.853114 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:33:50.853114 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:50.853761 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 427/840] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Sep 22 02:33:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:50.860652 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76992a-f7f6-4e16-9796-41c8c1d45f8a None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:51.073239 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76992a-f7f6-4e16-9796-41c8c1d45f8a None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:51.146941 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc84fa7-28da-4da0-99eb-e56a33cfdf0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:51.148219 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc84fa7-28da-4da0-99eb-e56a33cfdf0b None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:51.148781 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc84fa7-28da-4da0-99eb-e56a33cfdf0b None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:51.149256 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc84fa7-28da-4da0-99eb-e56a33cfdf0b None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:51.150537 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc84fa7-28da-4da0-99eb-e56a33cfdf0b None admin] RBAC: Adding query filter params (name=service) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:51.151113 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc84fa7-28da-4da0-99eb-e56a33cfdf0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:51.153779 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc84fa7-28da-4da0-99eb-e56a33cfdf0b None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:51.171730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 414/841] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Sep 22 02:33:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:33:51.178214 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ba2f1-977c-4269-98a7-e5cfcc8a84cf None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:51.351941 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ba2f1-977c-4269-98a7-e5cfcc8a84cf None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:51.405962 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc9164-24b4-475b-9dd7-af3fcea01ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:51.406636 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc9164-24b4-475b-9dd7-af3fcea01ee5 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:51.406965 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc9164-24b4-475b-9dd7-af3fcea01ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:51.407222 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc9164-24b4-475b-9dd7-af3fcea01ee5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:51.410460 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:51.419845 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc9164-24b4-475b-9dd7-af3fcea01ee5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:51.421396 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc9164-24b4-475b-9dd7-af3fcea01ee5 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:51.439854 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-c5bc9164-24b4-475b-9dd7-af3fcea01ee5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:51.439854 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:51.439854 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:51.439854 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:51.439854 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:51.439854 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:51.439854 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:51.439854 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:51.439854 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:51.439854 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:51.439854 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:51.439854 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:51.439854 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:51.439854 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:51.439854 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:51.439854 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:51.440896 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:51.440896 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:51.440896 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:51.440896 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:51.440896 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:51.440896 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:51.440896 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:51.440896 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:51.440896 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:51.440896 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:51.440896 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:51.440896 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:51.440896 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:51.440896 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:51.440896 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:51.440896 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:51.440896 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:51.441844 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:51.441844 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:51.441844 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:51.441844 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 428/842] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:51.448847 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc84fa7-28da-4da0-99eb-e56a33cfdf0b None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:51.505455 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:51.555299 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f574235-f912-4f9e-a63f-80d33148ac8a None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:51.556406 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f574235-f912-4f9e-a63f-80d33148ac8a None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:51.556921 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f574235-f912-4f9e-a63f-80d33148ac8a None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:51.558912 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 429/843] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:33:51.567144 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29157625-6704-4798-9971-67fc7c11327d None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:51.568057 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29157625-6704-4798-9971-67fc7c11327d None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:51.568694 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29157625-6704-4798-9971-67fc7c11327d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:51.591579 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-29157625-6704-4798-9971-67fc7c11327d None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:51.592858 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-29157625-6704-4798-9971-67fc7c11327d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:51.641940 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc84fa7-28da-4da0-99eb-e56a33cfdf0b None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:51.682798 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af974bdc-e931-44fa-869b-86d9a4f5037e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:51.683462 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af974bdc-e931-44fa-869b-86d9a4f5037e None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:51.683748 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af974bdc-e931-44fa-869b-86d9a4f5037e None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:51.684026 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af974bdc-e931-44fa-869b-86d9a4f5037e None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:51.684715 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af974bdc-e931-44fa-869b-86d9a4f5037e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:51.685028 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af974bdc-e931-44fa-869b-86d9a4f5037e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:51.686508 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af974bdc-e931-44fa-869b-86d9a4f5037e None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:51.700999 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 415/844] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:51.707511 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af974bdc-e931-44fa-869b-86d9a4f5037e None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:51.709364 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 430/845] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:51 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:33:51.823473 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:51.827180 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af974bdc-e931-44fa-869b-86d9a4f5037e None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:51.867421 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70332541-cf4f-43eb-ba23-73bab9b0c60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:51.868484 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70332541-cf4f-43eb-ba23-73bab9b0c60c None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:51.868484 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70332541-cf4f-43eb-ba23-73bab9b0c60c None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:51.868772 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70332541-cf4f-43eb-ba23-73bab9b0c60c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:51.876727 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70332541-cf4f-43eb-ba23-73bab9b0c60c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:51.878201 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70332541-cf4f-43eb-ba23-73bab9b0c60c None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:51.891236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-70332541-cf4f-43eb-ba23-73bab9b0c60c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 02:33:51.891236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:51.891236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:51.891236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:51.891236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:51.891236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:51.891236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:51.891236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:51.891236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:51.891236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:51.891236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:51.891236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:51.891236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:33:51.891236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:33:51.891236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:33:51.891236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:33:51.893079 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:51.893079 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:51.893079 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:33:51.893079 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:51.893079 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:33:51.893079 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:51.893079 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:51.893079 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:51.893079 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:51.893079 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:51.893079 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:51.893079 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:51.893079 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:51.893079 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:51.893079 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:51.893079 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:51.893079 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:51.893079 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:51.894984 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:51.894984 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:51.894984 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:33:51.894984 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:33:51.894984 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:33:51.894984 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:33:51.894984 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:33:51.894984 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:33:51.894984 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 02:33:51.894984 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:51.894984 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 416/846] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Sep 22 02:33:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:51.899775 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29157625-6704-4798-9971-67fc7c11327d None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:52.008086 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70332541-cf4f-43eb-ba23-73bab9b0c60c None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:52.056088 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29157625-6704-4798-9971-67fc7c11327d None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:52.081069 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:52.104273 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117b848-b7c0-4cd3-bb9b-5b679107ef4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:52.104904 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117b848-b7c0-4cd3-bb9b-5b679107ef4b None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:52.105182 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117b848-b7c0-4cd3-bb9b-5b679107ef4b None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:52.105445 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117b848-b7c0-4cd3-bb9b-5b679107ef4b None admin] PATH_INFO: `/v3/users` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:52.106242 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b117b848-b7c0-4cd3-bb9b-5b679107ef4b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:52.106554 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b117b848-b7c0-4cd3-bb9b-5b679107ef4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:52.108047 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b117b848-b7c0-4cd3-bb9b-5b679107ef4b None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:52.135620 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 431/847] 10.222.0.20 () {56 vars in 1149 bytes} [Thu Sep 22 02:33:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:52.143006 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117b848-b7c0-4cd3-bb9b-5b679107ef4b None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:52.188444 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70332541-cf4f-43eb-ba23-73bab9b0c60c None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:52.231332 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae2d08f-c246-473d-8fa0-1adf8016b672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:52.232465 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2d08f-c246-473d-8fa0-1adf8016b672 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:52.233285 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2d08f-c246-473d-8fa0-1adf8016b672 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:52.233778 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2d08f-c246-473d-8fa0-1adf8016b672 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:52.234732 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae2d08f-c246-473d-8fa0-1adf8016b672 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:52.236373 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae2d08f-c246-473d-8fa0-1adf8016b672 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:52.246609 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-fae2d08f-c246-473d-8fa0-1adf8016b672 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 02:33:52.246609 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:52.246609 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 02:33:52.246609 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 02:33:52.246609 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 02:33:52.246609 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 02:33:52.246609 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 02:33:52.246609 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:33:52.246609 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 02:33:52.246609 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:52.246609 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 02:33:52.246609 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:52.246609 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:52.246609 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:52.246609 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:52.246609 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:52.246609 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:52.247784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:52.247784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:52.247784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:52.247784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:52.247784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:52.247784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:52.247784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 02:33:52.247784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 02:33:52.247784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 02:33:52.247784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 02:33:52.247784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:52.247784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:52.247784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:52.247784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:52.247784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:52.247784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:52.248655 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:52.248655 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:52.248655 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:52.248655 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:52.248655 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:52.248655 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:52.248655 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:52.248655 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:52.248655 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:52.248655 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:52.248655 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 02:33:52.248655 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:33:52.248655 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 02:33:52.248655 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:52.251254 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 417/848] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Sep 22 02:33:52 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:33:52.261805 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae2d08f-c246-473d-8fa0-1adf8016b672 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:52.286475 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117b848-b7c0-4cd3-bb9b-5b679107ef4b None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:52.297700 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:52.345213 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5862244-ceda-400e-b1e1-fee4444bb1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:52.345878 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5862244-ceda-400e-b1e1-fee4444bb1ac None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:52.346164 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5862244-ceda-400e-b1e1-fee4444bb1ac None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:52.346440 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5862244-ceda-400e-b1e1-fee4444bb1ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:52.405272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:52.413503 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae2d08f-c246-473d-8fa0-1adf8016b672 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:52.435463 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5862244-ceda-400e-b1e1-fee4444bb1ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:52.436951 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5862244-ceda-400e-b1e1-fee4444bb1ac None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:52.461242 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a648de7a-bda7-49ed-a771-e8557fbb3584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:52.462271 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648de7a-bda7-49ed-a771-e8557fbb3584 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:52.463403 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648de7a-bda7-49ed-a771-e8557fbb3584 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:52.463994 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648de7a-bda7-49ed-a771-e8557fbb3584 None admin] PATH_INFO: `/v3/services` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:52.465258 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a648de7a-bda7-49ed-a771-e8557fbb3584 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:52.465832 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a648de7a-bda7-49ed-a771-e8557fbb3584 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:52.468445 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a648de7a-bda7-49ed-a771-e8557fbb3584 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:52.476748 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 418/849] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Sep 22 02:33:52 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:52.487313 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a648de7a-bda7-49ed-a771-e8557fbb3584 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:52.580624 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 432/850] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:33:52 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 02:33:52.581810 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5862244-ceda-400e-b1e1-fee4444bb1ac None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:52.622865 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:52.638850 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a648de7a-bda7-49ed-a771-e8557fbb3584 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:52.672146 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:52.678384 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef963ae-41a7-4810-9372-814dbc11a809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:52.679045 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef963ae-41a7-4810-9372-814dbc11a809 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:52.679333 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef963ae-41a7-4810-9372-814dbc11a809 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:52.679611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef963ae-41a7-4810-9372-814dbc11a809 None admin] PATH_INFO: `/v3/services` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:52.680266 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef963ae-41a7-4810-9372-814dbc11a809 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:52.680567 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef963ae-41a7-4810-9372-814dbc11a809 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:52.681949 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef963ae-41a7-4810-9372-814dbc11a809 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:52.691507 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 419/851] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Sep 22 02:33:52 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:33:52.691507 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe95cd-8939-41e3-9729-92b47f09f611 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:52.691507 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe95cd-8939-41e3-9729-92b47f09f611 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:52.691507 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe95cd-8939-41e3-9729-92b47f09f611 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:52.691507 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 420/852] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:33:52.698875 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500a3d0-97df-466f-9b0b-7e06dd860395 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:52.698875 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500a3d0-97df-466f-9b0b-7e06dd860395 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:52.698875 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500a3d0-97df-466f-9b0b-7e06dd860395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:52.709049 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5862244-ceda-400e-b1e1-fee4444bb1ac None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:52.717789 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-d500a3d0-97df-466f-9b0b-7e06dd860395 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:52.719100 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-d500a3d0-97df-466f-9b0b-7e06dd860395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:52.749017 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d021ab0-1a57-4ab6-9503-97ec0d6ac8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:52.749668 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d021ab0-1a57-4ab6-9503-97ec0d6ac8de None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:52.749958 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d021ab0-1a57-4ab6-9503-97ec0d6ac8de None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:52.750232 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d021ab0-1a57-4ab6-9503-97ec0d6ac8de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:52.761972 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d021ab0-1a57-4ab6-9503-97ec0d6ac8de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:52.763587 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d021ab0-1a57-4ab6-9503-97ec0d6ac8de None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:52.782284 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-6d021ab0-1a57-4ab6-9503-97ec0d6ac8de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:52.782284 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:52.782284 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:52.782284 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:52.782284 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:52.782284 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:52.782284 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:52.782284 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:52.782284 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:52.782284 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:52.782284 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:52.782284 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:52.782284 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:52.782284 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:52.782284 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:52.782284 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:52.783418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:52.783418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:52.783418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:52.783418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:52.783418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:52.783418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:52.783418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:52.783418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:52.783418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:52.783418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:52.783418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:52.783418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:52.783418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:52.783418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:52.783418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:52.783418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:52.783418 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:52.784347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:52.784347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:52.784347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:52.784347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 433/853] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:52.790794 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d021ab0-1a57-4ab6-9503-97ec0d6ac8de None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:52.830852 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 421/854] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 02:33:52.912372 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d021ab0-1a57-4ab6-9503-97ec0d6ac8de None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:52.940800 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:52.950440 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48aa533-737e-4809-83c7-6b367ed186fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:52.951097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48aa533-737e-4809-83c7-6b367ed186fb None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:52.951380 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48aa533-737e-4809-83c7-6b367ed186fb None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:52.951654 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48aa533-737e-4809-83c7-6b367ed186fb None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:52.952315 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48aa533-737e-4809-83c7-6b367ed186fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:52.952622 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48aa533-737e-4809-83c7-6b367ed186fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:52.953996 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48aa533-737e-4809-83c7-6b367ed186fb None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:52.967154 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 434/855] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:33:52.976072 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d500a3d0-97df-466f-9b0b-7e06dd860395 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:53.073566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:53.094355 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d500a3d0-97df-466f-9b0b-7e06dd860395 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:53.133491 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48aa533-737e-4809-83c7-6b367ed186fb None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:53.134427 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5fb364-c0ce-4681-98f7-8e28659e8a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:53.135128 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5fb364-c0ce-4681-98f7-8e28659e8a28 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:53.135413 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5fb364-c0ce-4681-98f7-8e28659e8a28 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:53.135692 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5fb364-c0ce-4681-98f7-8e28659e8a28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:53.151775 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5fb364-c0ce-4681-98f7-8e28659e8a28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:53.154195 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5fb364-c0ce-4681-98f7-8e28659e8a28 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:53.171628 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-de5fb364-c0ce-4681-98f7-8e28659e8a28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:53.171628 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:53.171628 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:53.171628 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:53.171628 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:53.171628 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:53.171628 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:53.171628 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:53.171628 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:53.171628 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:53.171628 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:53.171628 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:53.171628 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:33:53.171628 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:33:53.171628 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:33:53.172639 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:33:53.172639 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:53.172639 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:53.172639 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:53.172639 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:53.172639 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:53.172639 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:53.172639 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:53.172639 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:53.172639 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:53.172639 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:53.172639 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:53.172639 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:53.172639 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:53.172639 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:53.172639 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:53.172639 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:53.173557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:33:53.173557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:33:53.173557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:33:53.173557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:33:53.173557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:33:53.173557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:33:53.173557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:53.173557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:53.174014 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 422/856] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:33:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:53.183815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5fb364-c0ce-4681-98f7-8e28659e8a28 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:53.258876 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48aa533-737e-4809-83c7-6b367ed186fb None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:53.292861 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5fb364-c0ce-4681-98f7-8e28659e8a28 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:53.297286 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f4f84-f35c-441f-9267-0f193080ce69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:53.297910 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f4f84-f35c-441f-9267-0f193080ce69 None admin] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:53.298191 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f4f84-f35c-441f-9267-0f193080ce69 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:53.298466 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f4f84-f35c-441f-9267-0f193080ce69 None admin] PATH_INFO: `/v3/services` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:53.299167 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0f4f84-f35c-441f-9267-0f193080ce69 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:53.300616 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0f4f84-f35c-441f-9267-0f193080ce69 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:53.323091 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 435/857] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Sep 22 02:33:53 2022] POST /identity/v3/services => generated 269 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:33:53.325570 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd03cf-ecd9-444e-ab2a-6c1d53d255bc None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:53.326091 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd03cf-ecd9-444e-ab2a-6c1d53d255bc None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:53.326592 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd03cf-ecd9-444e-ab2a-6c1d53d255bc None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:53.331108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 436/858] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:33:53.339772 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e300db-7237-4fe0-9699-f6962c6ec94d None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:53.340434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e300db-7237-4fe0-9699-f6962c6ec94d None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:53.340628 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e300db-7237-4fe0-9699-f6962c6ec94d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:53.361778 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-93e300db-7237-4fe0-9699-f6962c6ec94d None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:53.363071 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-93e300db-7237-4fe0-9699-f6962c6ec94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:53.370537 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfeb1ce-7891-4182-92f2-0edf0a187615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:53.372180 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfeb1ce-7891-4182-92f2-0edf0a187615 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:53.372996 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfeb1ce-7891-4182-92f2-0edf0a187615 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:53.373781 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfeb1ce-7891-4182-92f2-0edf0a187615 None admin] PATH_INFO: `/v3/projects` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:53.375497 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfeb1ce-7891-4182-92f2-0edf0a187615 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:53.376383 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfeb1ce-7891-4182-92f2-0edf0a187615 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:53.380311 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfeb1ce-7891-4182-92f2-0edf0a187615 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:53.395973 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:53.408306 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 423/859] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Sep 22 02:33:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:53.414758 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfeb1ce-7891-4182-92f2-0edf0a187615 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:53.429983 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:53.466701 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 437/860] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:53 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 02:33:53.549687 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfeb1ce-7891-4182-92f2-0edf0a187615 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:53.588349 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc629743-2b1d-4cfb-9c3c-73f491415a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:53.589222 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc629743-2b1d-4cfb-9c3c-73f491415a46 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:53.589659 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc629743-2b1d-4cfb-9c3c-73f491415a46 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:53.590082 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc629743-2b1d-4cfb-9c3c-73f491415a46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:53.591029 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc629743-2b1d-4cfb-9c3c-73f491415a46 None admin] RBAC: Adding query filter params (role.id=fb2fbee669e14ffbbe0aa70642eab1ce, scope.project.id=2a792f8dc45145d687ddd58c94d44b5e, user.id=9821de2a961f4ddab2fca8e45b74e3db) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:53.591513 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc629743-2b1d-4cfb-9c3c-73f491415a46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:53.593136 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc629743-2b1d-4cfb-9c3c-73f491415a46 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:53.601062 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 424/861] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:33:53 2022] GET /identity/v3/role_assignments?user.id=9821de2a961f4ddab2fca8e45b74e3db&scope.project.id=2a792f8dc45145d687ddd58c94d44b5e&role.id=fb2fbee669e14ffbbe0aa70642eab1ce => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:53.984754 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e300db-7237-4fe0-9699-f6962c6ec94d None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:54.124584 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e300db-7237-4fe0-9699-f6962c6ec94d None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:54.163656 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24880115-5016-4e1f-b4a2-d33fa3c9bd2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:54.164679 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24880115-5016-4e1f-b4a2-d33fa3c9bd2f None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:54.165187 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24880115-5016-4e1f-b4a2-d33fa3c9bd2f None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:54.165698 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24880115-5016-4e1f-b4a2-d33fa3c9bd2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:54.178368 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24880115-5016-4e1f-b4a2-d33fa3c9bd2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:33:54.183786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-24880115-5016-4e1f-b4a2-d33fa3c9bd2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:33:54.183786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:54.183786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:54.183786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:54.183786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:54.183786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:54.183786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:54.183786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:54.183786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:54.183786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:54.183786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:54.183786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:54.183786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:33:54.183786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:33:54.183786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:33:54.183786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:33:54.184799 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:33:54.184799 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:33:54.184799 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:33:54.184799 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:33:54.184799 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:54.184799 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:54.184799 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:54.184799 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:54.184799 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:54.184799 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:54.184799 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:54.184799 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:54.184799 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:54.184799 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:54.184799 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:54.184799 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:54.185663 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:54.185663 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:54.185663 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:54.185663 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:54.185663 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:33:54.185663 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:33:54.185663 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:33:54.185663 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:33:54.185663 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:33:54.185663 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:33:54.185663 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:33:54.185663 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:54.188502 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 438/862] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Sep 22 02:33:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:54.198607 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc629743-2b1d-4cfb-9c3c-73f491415a46 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:54.273959 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:54.319645 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc629743-2b1d-4cfb-9c3c-73f491415a46 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:54.361894 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5e1127-7e39-49b4-bb71-ac4f222d29a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:54.362937 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e1127-7e39-49b4-bb71-ac4f222d29a2 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:54.363450 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e1127-7e39-49b4-bb71-ac4f222d29a2 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:54.364506 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e1127-7e39-49b4-bb71-ac4f222d29a2 None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:54.365420 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5e1127-7e39-49b4-bb71-ac4f222d29a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:54.365949 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5e1127-7e39-49b4-bb71-ac4f222d29a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:54.367603 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5e1127-7e39-49b4-bb71-ac4f222d29a2 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:54.378975 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 425/863] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:33:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:54.388883 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24880115-5016-4e1f-b4a2-d33fa3c9bd2f None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:54.472186 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:54.540281 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24880115-5016-4e1f-b4a2-d33fa3c9bd2f None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:54.579654 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d2c1e-5832-4ead-a231-d71cfdcbc2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:54.580317 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d2c1e-5832-4ead-a231-d71cfdcbc2e5 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:54.580600 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d2c1e-5832-4ead-a231-d71cfdcbc2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:54.580876 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d2c1e-5832-4ead-a231-d71cfdcbc2e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:54.593073 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88d2c1e-5832-4ead-a231-d71cfdcbc2e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:54.594606 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88d2c1e-5832-4ead-a231-d71cfdcbc2e5 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:54.614699 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-b88d2c1e-5832-4ead-a231-d71cfdcbc2e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:54.614699 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:54.614699 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:54.614699 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:54.614699 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:54.614699 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:54.614699 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:54.614699 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:54.614699 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:54.614699 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:54.614699 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:54.614699 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:54.614699 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:54.614699 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:54.614699 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:54.614699 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:54.615773 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:54.615773 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:54.615773 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:54.615773 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:54.615773 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:54.615773 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:54.615773 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:54.615773 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:54.615773 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:54.615773 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:54.615773 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:54.615773 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:54.615773 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:54.615773 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:54.615773 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:54.615773 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:54.615773 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:54.616672 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:54.616672 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:54.616672 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:54.616672 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 439/864] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:54.625432 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5e1127-7e39-49b4-bb71-ac4f222d29a2 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:54.774797 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5e1127-7e39-49b4-bb71-ac4f222d29a2 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:54.802083 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:54.815071 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1c898-e544-4e74-b93b-2df099fe0a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:54.816085 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1c898-e544-4e74-b93b-2df099fe0a39 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:54.816572 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1c898-e544-4e74-b93b-2df099fe0a39 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:54.817081 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1c898-e544-4e74-b93b-2df099fe0a39 None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:54.817963 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd1c898-e544-4e74-b93b-2df099fe0a39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:54.818490 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd1c898-e544-4e74-b93b-2df099fe0a39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:54.820175 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd1c898-e544-4e74-b93b-2df099fe0a39 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:54.837719 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 426/865] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:54.846790 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d2c1e-5832-4ead-a231-d71cfdcbc2e5 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:54.985099 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:54.996247 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d2c1e-5832-4ead-a231-d71cfdcbc2e5 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:55.055895 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b8e7cb-010e-4d3c-ba9f-5d8a7adf2cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:55.056940 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8e7cb-010e-4d3c-ba9f-5d8a7adf2cc1 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:55.057384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8e7cb-010e-4d3c-ba9f-5d8a7adf2cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:55.057875 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8e7cb-010e-4d3c-ba9f-5d8a7adf2cc1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:55.072154 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b8e7cb-010e-4d3c-ba9f-5d8a7adf2cc1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:55.074880 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b8e7cb-010e-4d3c-ba9f-5d8a7adf2cc1 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:55.097994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-06b8e7cb-010e-4d3c-ba9f-5d8a7adf2cc1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 22 02:33:55.097994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:55.097994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:55.097994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:55.097994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:55.097994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:55.097994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:55.097994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:55.097994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:55.097994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:55.097994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:55.097994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:55.097994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:33:55.097994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:33:55.097994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:33:55.097994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:33:55.099636 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:55.099636 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:55.099636 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:33:55.099636 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:55.099636 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:33:55.099636 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:55.099636 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:55.099636 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:55.099636 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:55.099636 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:55.099636 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:55.099636 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:55.099636 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:55.099636 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:55.099636 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:55.099636 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:55.099636 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:55.099636 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:55.100572 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:55.100572 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:55.100572 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:33:55.100572 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:33:55.100572 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:33:55.100572 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:33:55.100572 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:33:55.100572 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:33:55.100572 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 22 02:33:55.100572 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:55.101610 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 440/866] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Sep 22 02:33:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:55.107994 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1c898-e544-4e74-b93b-2df099fe0a39 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:55.225828 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1c898-e544-4e74-b93b-2df099fe0a39 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:55.262934 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc6a68-9aa8-4ebd-971b-25e694814fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:55.263957 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc6a68-9aa8-4ebd-971b-25e694814fe5 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:55.264454 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc6a68-9aa8-4ebd-971b-25e694814fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:55.264953 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc6a68-9aa8-4ebd-971b-25e694814fe5 None admin] PATH_INFO: `/v3/users` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:55.265913 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dc6a68-9aa8-4ebd-971b-25e694814fe5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:55.266428 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dc6a68-9aa8-4ebd-971b-25e694814fe5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:55.268134 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dc6a68-9aa8-4ebd-971b-25e694814fe5 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:55.288839 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 427/867] 10.222.0.20 () {56 vars in 1135 bytes} [Thu Sep 22 02:33:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:33:55.298869 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b8e7cb-010e-4d3c-ba9f-5d8a7adf2cc1 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:55.303588 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327f49d-67ae-4fdc-b445-31df30fee1ed None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:55.304100 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327f49d-67ae-4fdc-b445-31df30fee1ed None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:55.304547 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327f49d-67ae-4fdc-b445-31df30fee1ed None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:55.306118 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 428/868] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:33:55.313694 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f152a-fdd4-4204-aa5c-95a5ad5c13f0 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:55.314048 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f152a-fdd4-4204-aa5c-95a5ad5c13f0 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:55.314254 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f152a-fdd4-4204-aa5c-95a5ad5c13f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:55.334193 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-bf6f152a-fdd4-4204-aa5c-95a5ad5c13f0 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:55.335466 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-bf6f152a-fdd4-4204-aa5c-95a5ad5c13f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:55.387304 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:55.414867 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 429/869] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:55 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 02:33:55.448990 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b8e7cb-010e-4d3c-ba9f-5d8a7adf2cc1 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:55.497553 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab7c0a-b741-47f9-88d6-ac936fbb25be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:55.498331 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab7c0a-b741-47f9-88d6-ac936fbb25be None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:55.498814 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab7c0a-b741-47f9-88d6-ac936fbb25be None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:55.499300 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab7c0a-b741-47f9-88d6-ac936fbb25be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:55.525392 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:55.594526 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab7c0a-b741-47f9-88d6-ac936fbb25be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:55.596041 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab7c0a-b741-47f9-88d6-ac936fbb25be None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:55.749325 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 441/870] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:33:55 2022] GET /identity/v3/auth/tokens => generated 1271 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 02:33:55.756000 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f152a-fdd4-4204-aa5c-95a5ad5c13f0 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:55.780688 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab7c0a-b741-47f9-88d6-ac936fbb25be None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:55.862627 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:55.887669 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f152a-fdd4-4204-aa5c-95a5ad5c13f0 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:55.929516 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf2607-0918-4948-80ff-c36a9bb269da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:55.930183 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf2607-0918-4948-80ff-c36a9bb269da None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:55.930463 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf2607-0918-4948-80ff-c36a9bb269da None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:55.930734 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf2607-0918-4948-80ff-c36a9bb269da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:55.942025 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bf2607-0918-4948-80ff-c36a9bb269da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:55.943543 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bf2607-0918-4948-80ff-c36a9bb269da None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:55.959516 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-01bf2607-0918-4948-80ff-c36a9bb269da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:55.959516 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:55.959516 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:55.959516 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:55.959516 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:55.959516 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:55.959516 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:55.959516 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:55.959516 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:55.959516 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:55.959516 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:55.959516 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:55.959516 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:55.959516 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:55.959516 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:55.959516 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:55.960566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:55.960566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:55.960566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:55.960566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:55.960566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:55.960566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:55.960566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:55.960566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:55.960566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:55.960566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:55.960566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:55.960566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:55.960566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:55.960566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:55.960566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:55.960566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:55.960566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:55.961480 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:55.961480 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:55.961480 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:55.962136 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab7c0a-b741-47f9-88d6-ac936fbb25be None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:55.963886 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 430/871] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:55.965035 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f880c40b-1c28-4f59-be9e-07dde61b4a0b None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:55.965420 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f880c40b-1c28-4f59-be9e-07dde61b4a0b None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:55.965704 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f880c40b-1c28-4f59-be9e-07dde61b4a0b None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:55.974081 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 431/872] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:33:55.974081 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f880c40b-1c28-4f59-be9e-07dde61b4a0b None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:56.002375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:56.033636 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d30c443-2838-465a-bec1-0c67e741e6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:56.034670 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d30c443-2838-465a-bec1-0c67e741e6d5 None admin] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:56.036251 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d30c443-2838-465a-bec1-0c67e741e6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:56.036675 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d30c443-2838-465a-bec1-0c67e741e6d5 None admin] PATH_INFO: `/v3/services` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:56.037643 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d30c443-2838-465a-bec1-0c67e741e6d5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:56.039610 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d30c443-2838-465a-bec1-0c67e741e6d5 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:56.059262 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 442/873] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Sep 22 02:33:55 2022] POST /identity/v3/services => generated 251 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 02:33:56.061482 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf503e-3fe6-439c-8e49-3e0a6891b75a None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:56.061973 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf503e-3fe6-439c-8e49-3e0a6891b75a None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:56.062455 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf503e-3fe6-439c-8e49-3e0a6891b75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:56.095182 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-86bf503e-3fe6-439c-8e49-3e0a6891b75a None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:56.097198 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-86bf503e-3fe6-439c-8e49-3e0a6891b75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:56.163496 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f880c40b-1c28-4f59-be9e-07dde61b4a0b None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:56.217645 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 443/874] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:56 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:33:56.219776 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1950e5-3c6d-415f-8685-6de2ca80d368 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:56.220379 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1950e5-3c6d-415f-8685-6de2ca80d368 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:56.221448 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1950e5-3c6d-415f-8685-6de2ca80d368 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:56.224265 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 444/875] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:33:56.232169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325cb022-17d9-4f73-b103-f005730a1b54 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:56.232916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325cb022-17d9-4f73-b103-f005730a1b54 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:56.233478 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325cb022-17d9-4f73-b103-f005730a1b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:56.234715 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72efc44-0056-4f9a-994c-2d886ac6caaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:56.235641 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72efc44-0056-4f9a-994c-2d886ac6caaa None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:56.236109 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72efc44-0056-4f9a-994c-2d886ac6caaa None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:56.236630 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72efc44-0056-4f9a-994c-2d886ac6caaa None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:56.237876 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72efc44-0056-4f9a-994c-2d886ac6caaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:56.238441 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72efc44-0056-4f9a-994c-2d886ac6caaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:56.241584 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72efc44-0056-4f9a-994c-2d886ac6caaa None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:56.258806 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-325cb022-17d9-4f73-b103-f005730a1b54 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:56.260559 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-325cb022-17d9-4f73-b103-f005730a1b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:56.268241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 432/876] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:56.271940 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72efc44-0056-4f9a-994c-2d886ac6caaa None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:56.324682 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:56.373961 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 445/877] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:56 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:33:56.399437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:56.422532 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72efc44-0056-4f9a-994c-2d886ac6caaa None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:56.476390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17581e6-346e-48c4-8338-804ae8c3eb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:56.477548 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17581e6-346e-48c4-8338-804ae8c3eb0e None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:56.477949 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17581e6-346e-48c4-8338-804ae8c3eb0e None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:56.478237 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17581e6-346e-48c4-8338-804ae8c3eb0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:56.491291 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17581e6-346e-48c4-8338-804ae8c3eb0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:56.493727 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17581e6-346e-48c4-8338-804ae8c3eb0e None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:56.515903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-d17581e6-346e-48c4-8338-804ae8c3eb0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:56.515903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:56.515903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:56.515903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:56.515903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:56.515903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:56.515903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:56.515903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:56.515903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:56.515903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:56.515903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:56.515903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:56.515903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:33:56.515903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:33:56.515903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:33:56.516863 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:33:56.516863 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:56.516863 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:56.516863 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:56.516863 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:56.516863 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:56.516863 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:56.516863 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:56.516863 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:56.516863 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:56.516863 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:56.516863 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:56.516863 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:56.516863 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:56.516863 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:56.516863 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:56.516863 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:56.517781 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:33:56.517781 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:33:56.517781 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:33:56.517781 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:33:56.517781 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:33:56.517781 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:33:56.517781 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:33:56.517781 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:56.518779 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 433/878] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:33:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:56.528267 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325cb022-17d9-4f73-b103-f005730a1b54 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:56.626630 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17581e6-346e-48c4-8338-804ae8c3eb0e None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:56.696288 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325cb022-17d9-4f73-b103-f005730a1b54 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:56.743902 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6ab831-ce85-4f7e-9615-e45b2eab300a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:56.744949 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ab831-ce85-4f7e-9615-e45b2eab300a None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:56.745448 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ab831-ce85-4f7e-9615-e45b2eab300a None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:56.745929 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ab831-ce85-4f7e-9615-e45b2eab300a None admin] PATH_INFO: `/v3/projects` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:56.747264 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6ab831-ce85-4f7e-9615-e45b2eab300a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:56.747822 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6ab831-ce85-4f7e-9615-e45b2eab300a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:56.750511 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6ab831-ce85-4f7e-9615-e45b2eab300a None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:56.774883 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 446/879] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Sep 22 02:33:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:56.776372 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6ab831-ce85-4f7e-9615-e45b2eab300a None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:56.776755 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17581e6-346e-48c4-8338-804ae8c3eb0e None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:56.815510 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba2be9-7fe5-4d47-b952-468f48556e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:56.816170 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba2be9-7fe5-4d47-b952-468f48556e3e None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:56.816457 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba2be9-7fe5-4d47-b952-468f48556e3e None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:56.817280 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba2be9-7fe5-4d47-b952-468f48556e3e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:56.817900 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba2be9-7fe5-4d47-b952-468f48556e3e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:56.819390 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba2be9-7fe5-4d47-b952-468f48556e3e None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:56.829465 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-dfba2be9-7fe5-4d47-b952-468f48556e3e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 02:33:56.829465 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:56.829465 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 02:33:56.829465 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 02:33:56.829465 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 02:33:56.829465 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 02:33:56.829465 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 02:33:56.829465 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:33:56.829465 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 02:33:56.829465 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:56.829465 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 02:33:56.829465 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:56.829465 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:56.829465 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:56.829465 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:56.829465 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:56.830565 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:56.830565 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:56.830565 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:56.830565 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:56.830565 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:56.830565 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:56.830565 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:56.830565 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 02:33:56.830565 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 02:33:56.830565 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 02:33:56.830565 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 02:33:56.830565 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:56.830565 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:56.830565 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:56.830565 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:56.830565 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:56.830565 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:56.831844 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:56.831844 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:56.831844 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:56.831844 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:56.831844 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:56.831844 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:56.831844 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:56.831844 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:56.831844 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:56.831844 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:56.831844 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 02:33:56.831844 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:33:56.831844 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 02:33:56.831844 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:56.831844 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 434/880] 10.222.0.20 () {56 vars in 1132 bytes} [Thu Sep 22 02:33:56 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:33:56.833257 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba2be9-7fe5-4d47-b952-468f48556e3e None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:56.837964 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:56.842106 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:56.962881 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba2be9-7fe5-4d47-b952-468f48556e3e None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:56.966011 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6ab831-ce85-4f7e-9615-e45b2eab300a None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:57.009096 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78975e2b-e084-4ce5-af31-f54efe3d3aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:57.009711 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78975e2b-e084-4ce5-af31-f54efe3d3aba None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:57.009989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78975e2b-e084-4ce5-af31-f54efe3d3aba None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:57.010258 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78975e2b-e084-4ce5-af31-f54efe3d3aba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:57.011112 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78975e2b-e084-4ce5-af31-f54efe3d3aba None admin] RBAC: Adding query filter params (role.id=31a687548f55493b8ea666fe7e350303, scope.project.id=2a792f8dc45145d687ddd58c94d44b5e, user.id=4b9ad548622b4e54a6fecd0afd5ef3c2) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:57.011333 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78975e2b-e084-4ce5-af31-f54efe3d3aba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:57.012804 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78975e2b-e084-4ce5-af31-f54efe3d3aba None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:57.022235 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 435/881] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:33:56 2022] GET /identity/v3/role_assignments?user.id=4b9ad548622b4e54a6fecd0afd5ef3c2&scope.project.id=2a792f8dc45145d687ddd58c94d44b5e&role.id=31a687548f55493b8ea666fe7e350303 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:57.022235 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78975e2b-e084-4ce5-af31-f54efe3d3aba None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:57.038680 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e000f2-f475-4fdf-8be0-ba68eb57f534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:57.040622 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e000f2-f475-4fdf-8be0-ba68eb57f534 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:57.041223 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e000f2-f475-4fdf-8be0-ba68eb57f534 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:57.041709 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e000f2-f475-4fdf-8be0-ba68eb57f534 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:57.059456 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e000f2-f475-4fdf-8be0-ba68eb57f534 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:33:57.068354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-e7e000f2-f475-4fdf-8be0-ba68eb57f534 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 02:33:57.068354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:57.068354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:57.068354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:57.068354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:57.068354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:57.068354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:57.068354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:57.068354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:57.068354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:57.068354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:57.068354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:57.068354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:33:57.068354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:33:57.068354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:33:57.068354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:33:57.069417 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:33:57.069417 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:33:57.069417 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:33:57.069417 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:33:57.069417 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:57.069417 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:57.069417 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:57.069417 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:57.069417 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:57.069417 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:57.069417 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:57.069417 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:57.069417 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:57.069417 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:57.069417 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:57.069417 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:57.070311 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:57.070311 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:57.070311 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:57.070311 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:57.070311 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:33:57.070311 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:33:57.070311 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:33:57.070311 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:33:57.070311 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:33:57.070311 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:33:57.070311 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 02:33:57.070311 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:57.080620 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 447/882] 10.222.0.20 () {56 vars in 1120 bytes} [Thu Sep 22 02:33:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:57.085111 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e000f2-f475-4fdf-8be0-ba68eb57f534 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:57.142904 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:57.196877 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78975e2b-e084-4ce5-af31-f54efe3d3aba None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:57.244716 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d80e20-31f0-4bfa-8230-a95fffcb588a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:57.246060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d80e20-31f0-4bfa-8230-a95fffcb588a None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:57.246503 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d80e20-31f0-4bfa-8230-a95fffcb588a None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:57.246982 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d80e20-31f0-4bfa-8230-a95fffcb588a None admin] PATH_INFO: `/v3/services` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:57.247825 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d80e20-31f0-4bfa-8230-a95fffcb588a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:57.248310 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d80e20-31f0-4bfa-8230-a95fffcb588a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:57.250759 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d80e20-31f0-4bfa-8230-a95fffcb588a None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:57.255648 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:57.260620 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 436/883] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Sep 22 02:33:57 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:57.268921 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d80e20-31f0-4bfa-8230-a95fffcb588a None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:57.271437 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e000f2-f475-4fdf-8be0-ba68eb57f534 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:57.320388 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf308f2-8bfc-4e74-a31e-f86d80fe0848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:57.321528 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf308f2-8bfc-4e74-a31e-f86d80fe0848 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:57.321997 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf308f2-8bfc-4e74-a31e-f86d80fe0848 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:57.322512 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf308f2-8bfc-4e74-a31e-f86d80fe0848 None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:57.323747 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf308f2-8bfc-4e74-a31e-f86d80fe0848 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:57.324246 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf308f2-8bfc-4e74-a31e-f86d80fe0848 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:57.326271 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf308f2-8bfc-4e74-a31e-f86d80fe0848 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:57.342212 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 448/884] 10.222.0.20 () {56 vars in 1115 bytes} [Thu Sep 22 02:33:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:57.351019 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf308f2-8bfc-4e74-a31e-f86d80fe0848 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:57.482982 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d80e20-31f0-4bfa-8230-a95fffcb588a None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:57.550704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf308f2-8bfc-4e74-a31e-f86d80fe0848 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:57.552864 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:57.564597 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73707392-a490-4de9-98f1-ec949c33ffed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:57.567523 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73707392-a490-4de9-98f1-ec949c33ffed None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:57.568341 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73707392-a490-4de9-98f1-ec949c33ffed None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:57.569097 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73707392-a490-4de9-98f1-ec949c33ffed None admin] PATH_INFO: `/v3/services` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:57.571362 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73707392-a490-4de9-98f1-ec949c33ffed None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:57.572167 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73707392-a490-4de9-98f1-ec949c33ffed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:57.575959 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73707392-a490-4de9-98f1-ec949c33ffed None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:57.595011 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 437/885] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Sep 22 02:33:57 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:33:57.597779 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73707392-a490-4de9-98f1-ec949c33ffed None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:57.616415 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f94c5-e441-498e-acb5-5aeed2ced86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:57.617475 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f94c5-e441-498e-acb5-5aeed2ced86d None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:57.617976 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f94c5-e441-498e-acb5-5aeed2ced86d None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:57.618471 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f94c5-e441-498e-acb5-5aeed2ced86d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:57.638138 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f94c5-e441-498e-acb5-5aeed2ced86d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:57.640787 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f94c5-e441-498e-acb5-5aeed2ced86d None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:57.669945 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-dc2f94c5-e441-498e-acb5-5aeed2ced86d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:57.669945 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:57.669945 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:57.669945 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:57.669945 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:57.669945 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:57.669945 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:57.669945 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:57.669945 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:57.669945 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:57.669945 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:57.669945 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:57.669945 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:57.669945 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:57.669945 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:57.669945 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:57.671911 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:57.671911 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:57.671911 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:57.671911 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:57.671911 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:57.671911 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:57.671911 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:57.671911 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:57.671911 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:57.671911 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:57.671911 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:57.671911 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:57.671911 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:57.671911 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:57.671911 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:57.671911 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:57.671911 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:57.674122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:57.674122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:57.674122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:57.678937 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 449/886] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:33:57.682076 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f94c5-e441-498e-acb5-5aeed2ced86d None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:57.690070 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:57.813891 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73707392-a490-4de9-98f1-ec949c33ffed None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:57.877471 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f94c5-e441-498e-acb5-5aeed2ced86d None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:57.883103 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d83fad1-fc91-4852-93aa-ea2f5e64b64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:57.884460 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d83fad1-fc91-4852-93aa-ea2f5e64b64c None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:57.885196 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d83fad1-fc91-4852-93aa-ea2f5e64b64c None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:57.885916 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d83fad1-fc91-4852-93aa-ea2f5e64b64c None admin] PATH_INFO: `/v3/endpoints` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:57.887551 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d83fad1-fc91-4852-93aa-ea2f5e64b64c None admin] RBAC: Adding query filter params (interface=public, service_id=10f691feab2f404188d6d0e49962a1b2, region_id=RegionOne) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:57.888427 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d83fad1-fc91-4852-93aa-ea2f5e64b64c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:57.891577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d83fad1-fc91-4852-93aa-ea2f5e64b64c None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:57.914479 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 438/887] 10.222.0.20 () {56 vars in 1251 bytes} [Thu Sep 22 02:33:57 2022] GET /identity/v3/endpoints?service_id=10f691feab2f404188d6d0e49962a1b2&interface=public®ion_id=RegionOne => generated 194 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:57.917603 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d83fad1-fc91-4852-93aa-ea2f5e64b64c None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:57.945713 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc4c1c1-bd30-44e5-9e26-d8ed781aa2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:57.946782 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4c1c1-bd30-44e5-9e26-d8ed781aa2a7 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:57.947334 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4c1c1-bd30-44e5-9e26-d8ed781aa2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:57.947839 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4c1c1-bd30-44e5-9e26-d8ed781aa2a7 None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:57.949162 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc4c1c1-bd30-44e5-9e26-d8ed781aa2a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:57.950150 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc4c1c1-bd30-44e5-9e26-d8ed781aa2a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:57.952454 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc4c1c1-bd30-44e5-9e26-d8ed781aa2a7 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:57.983278 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 450/888] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:33:57.987724 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc4c1c1-bd30-44e5-9e26-d8ed781aa2a7 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:58.070389 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:58.128817 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:58.155546 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d83fad1-fc91-4852-93aa-ea2f5e64b64c None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:58.217436 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc4c1c1-bd30-44e5-9e26-d8ed781aa2a7 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:58.221777 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410bb389-5603-4562-bbc3-a9ed6eb79b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:58.222811 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410bb389-5603-4562-bbc3-a9ed6eb79b7d None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:58.224629 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410bb389-5603-4562-bbc3-a9ed6eb79b7d None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:58.224629 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410bb389-5603-4562-bbc3-a9ed6eb79b7d None admin] PATH_INFO: `/v3/services` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:58.225102 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410bb389-5603-4562-bbc3-a9ed6eb79b7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:58.227871 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410bb389-5603-4562-bbc3-a9ed6eb79b7d None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:58.245818 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 439/889] 10.222.0.20 () {56 vars in 1087 bytes} [Thu Sep 22 02:33:57 2022] GET /identity/v3/services => generated 1268 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 02:33:58.299880 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab9b78e-bbab-4f81-81e6-324004f96265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:58.301574 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9b78e-bbab-4f81-81e6-324004f96265 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:58.302084 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9b78e-bbab-4f81-81e6-324004f96265 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:58.302576 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9b78e-bbab-4f81-81e6-324004f96265 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:58.318503 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab9b78e-bbab-4f81-81e6-324004f96265 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:58.321520 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab9b78e-bbab-4f81-81e6-324004f96265 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:58.346435 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-bab9b78e-bbab-4f81-81e6-324004f96265 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 02:33:58.346435 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:58.346435 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:58.346435 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:58.346435 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:58.346435 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:58.346435 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:58.346435 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:58.346435 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:58.346435 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:58.346435 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:58.346435 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:58.346435 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:33:58.346435 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:33:58.346435 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:33:58.346435 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:33:58.348336 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:58.348336 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:58.348336 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:33:58.348336 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:58.348336 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:33:58.348336 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:33:58.348336 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:58.348336 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:58.348336 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:58.348336 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:58.348336 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:58.348336 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:58.348336 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:58.348336 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:58.348336 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:58.348336 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:58.348336 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:58.348336 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:58.350919 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:58.350919 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:58.350919 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:33:58.350919 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:33:58.350919 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:33:58.350919 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:33:58.350919 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:33:58.350919 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:33:58.350919 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 02:33:58.350919 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:58.355086 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 451/890] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Sep 22 02:33:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:33:58.364786 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410bb389-5603-4562-bbc3-a9ed6eb79b7d None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:58.565237 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410bb389-5603-4562-bbc3-a9ed6eb79b7d None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:58.623478 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:58.656275 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c67adb4-a71e-4e91-be08-60031b3c97d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:58.657841 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67adb4-a71e-4e91-be08-60031b3c97d4 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:58.658594 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67adb4-a71e-4e91-be08-60031b3c97d4 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:58.659375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67adb4-a71e-4e91-be08-60031b3c97d4 None admin] PATH_INFO: `/v3/users` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:58.661704 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c67adb4-a71e-4e91-be08-60031b3c97d4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:58.662473 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c67adb4-a71e-4e91-be08-60031b3c97d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:58.665314 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c67adb4-a71e-4e91-be08-60031b3c97d4 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:58.700480 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 440/891] 10.222.0.20 () {56 vars in 1149 bytes} [Thu Sep 22 02:33:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:33:58.709943 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab9b78e-bbab-4f81-81e6-324004f96265 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:58.898929 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d14d8-b9b4-489a-9ce0-db5e6e2ee555 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:58.902220 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d14d8-b9b4-489a-9ce0-db5e6e2ee555 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:58.902220 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d14d8-b9b4-489a-9ce0-db5e6e2ee555 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:58.911479 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 441/892] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:33:58 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:33:58.922556 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d884abe-f9d3-41ca-9f4d-07bcf02e2e40 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:58.924016 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d884abe-f9d3-41ca-9f4d-07bcf02e2e40 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:58.924754 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d884abe-f9d3-41ca-9f4d-07bcf02e2e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:58.973539 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-7d884abe-f9d3-41ca-9f4d-07bcf02e2e40 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:33:58.976771 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-7d884abe-f9d3-41ca-9f4d-07bcf02e2e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:33:58.988188 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab9b78e-bbab-4f81-81e6-324004f96265 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:59.051945 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:59.080031 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a3ec3-861a-4304-9284-06375f8f38d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:59.080673 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a3ec3-861a-4304-9284-06375f8f38d9 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:59.080960 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a3ec3-861a-4304-9284-06375f8f38d9 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:59.081236 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a3ec3-861a-4304-9284-06375f8f38d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:59.225369 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 442/893] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:33:58 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:33:59.268991 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a3ec3-861a-4304-9284-06375f8f38d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:59.271131 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a3ec3-861a-4304-9284-06375f8f38d9 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:59.381967 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 452/894] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:33:58 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 673 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 02:33:59.387470 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d884abe-f9d3-41ca-9f4d-07bcf02e2e40 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:59.606226 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a3ec3-861a-4304-9284-06375f8f38d9 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:59.618242 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d884abe-f9d3-41ca-9f4d-07bcf02e2e40 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:59.654466 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:59.662437 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:33:59.707121 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51950f-30c3-451e-827e-a3617169ae6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:59.708808 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51950f-30c3-451e-827e-a3617169ae6c None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:59.708983 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51950f-30c3-451e-827e-a3617169ae6c None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:59.709463 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51950f-30c3-451e-827e-a3617169ae6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:59.731038 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc51950f-30c3-451e-827e-a3617169ae6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:59.732858 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc51950f-30c3-451e-827e-a3617169ae6c None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:59.750730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-cc51950f-30c3-451e-827e-a3617169ae6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:59.750730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:59.750730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:59.750730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:59.750730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:59.750730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:59.750730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:59.750730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:59.750730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:59.750730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:59.750730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:59.750730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:59.750730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:33:59.750730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:33:59.750730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:33:59.750730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:33:59.752085 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:59.752085 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:59.752085 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:59.752085 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:59.752085 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:59.752085 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:59.752085 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:59.752085 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:59.752085 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:59.752085 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:59.752085 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:59.752085 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:59.752085 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:59.752085 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:59.752085 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:59.752085 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:59.752085 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:33:59.753059 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:33:59.753059 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:33:59.753059 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:33:59.753059 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 443/895] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:33:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:59.758490 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51950f-30c3-451e-827e-a3617169ae6c None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:59.806869 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a3ec3-861a-4304-9284-06375f8f38d9 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:59.846467 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9233369c-4919-4b15-8469-d6556d66625b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:59.848461 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233369c-4919-4b15-8469-d6556d66625b None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:59.848965 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233369c-4919-4b15-8469-d6556d66625b None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:59.849438 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233369c-4919-4b15-8469-d6556d66625b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:59.850588 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9233369c-4919-4b15-8469-d6556d66625b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:59.852406 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9233369c-4919-4b15-8469-d6556d66625b None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:33:59.863387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-9233369c-4919-4b15-8469-d6556d66625b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 02:33:59.863387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:59.863387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 02:33:59.863387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 02:33:59.863387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 02:33:59.863387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 02:33:59.863387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 02:33:59.863387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:33:59.863387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 02:33:59.863387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:59.863387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 02:33:59.863387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:59.863387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:33:59.863387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:33:59.863387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:33:59.863387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:33:59.863387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:33:59.865011 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:33:59.865011 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:33:59.865011 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:33:59.865011 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:33:59.865011 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:33:59.865011 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:33:59.865011 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 02:33:59.865011 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 02:33:59.865011 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 02:33:59.865011 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 02:33:59.865011 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:33:59.865011 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:33:59.865011 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:33:59.865011 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:33:59.865011 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:33:59.865011 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:33:59.865920 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:33:59.865920 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:33:59.865920 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:33:59.865920 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:33:59.865920 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:33:59.865920 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:33:59.865920 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:33:59.865920 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:33:59.865920 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:33:59.865920 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:33:59.865920 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 02:33:59.865920 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:33:59.865920 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 02:33:59.865920 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:33:59.868757 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 453/896] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Sep 22 02:33:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:33:59.879430 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9233369c-4919-4b15-8469-d6556d66625b None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:33:59.940152 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51950f-30c3-451e-827e-a3617169ae6c None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:33:59.983571 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec41cb-2cd1-43f1-8222-c2243aa9a128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:33:59.984687 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec41cb-2cd1-43f1-8222-c2243aa9a128 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:33:59.985041 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec41cb-2cd1-43f1-8222-c2243aa9a128 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:33:59.985323 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec41cb-2cd1-43f1-8222-c2243aa9a128 None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:33:59.985991 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ec41cb-2cd1-43f1-8222-c2243aa9a128 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:33:59.986299 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ec41cb-2cd1-43f1-8222-c2243aa9a128 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:33:59.988098 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ec41cb-2cd1-43f1-8222-c2243aa9a128 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:00.005361 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 444/897] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:33:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:00.014409 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec41cb-2cd1-43f1-8222-c2243aa9a128 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:00.058102 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:00.065049 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:00.084425 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9233369c-4919-4b15-8469-d6556d66625b None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:00.147665 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4ac38-e11e-4253-a3b5-3c1ebe670542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:00.148254 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4ac38-e11e-4253-a3b5-3c1ebe670542 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:00.148540 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4ac38-e11e-4253-a3b5-3c1ebe670542 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:00.148812 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4ac38-e11e-4253-a3b5-3c1ebe670542 None admin] PATH_INFO: `/v3/services` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:00.149478 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4ac38-e11e-4253-a3b5-3c1ebe670542 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:00.149787 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4ac38-e11e-4253-a3b5-3c1ebe670542 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:00.151224 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4ac38-e11e-4253-a3b5-3c1ebe670542 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:00.158632 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 454/898] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Sep 22 02:33:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:34:00.167639 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4ac38-e11e-4253-a3b5-3c1ebe670542 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:00.211846 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec41cb-2cd1-43f1-8222-c2243aa9a128 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:00.284956 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c53bf-f9fa-41ca-ba02-2882e430187d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:00.287114 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c53bf-f9fa-41ca-ba02-2882e430187d None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:00.287894 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c53bf-f9fa-41ca-ba02-2882e430187d None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:00.288612 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c53bf-f9fa-41ca-ba02-2882e430187d None admin] PATH_INFO: `/v3/projects/service` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:00.302745 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2c53bf-f9fa-41ca-ba02-2882e430187d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:00.306025 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2c53bf-f9fa-41ca-ba02-2882e430187d None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:00.337446 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-9f2c53bf-f9fa-41ca-ba02-2882e430187d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:34:00.337446 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:00.337446 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:00.337446 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:00.337446 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:00.337446 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:00.337446 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:00.337446 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:00.337446 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:00.337446 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:00.337446 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:00.337446 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:00.337446 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:34:00.337446 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:34:00.337446 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:34:00.339464 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:34:00.339464 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:00.339464 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:00.339464 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:00.339464 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:00.339464 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:00.339464 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:00.339464 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:00.339464 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:00.339464 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:00.339464 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:00.339464 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:00.339464 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:00.339464 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:00.339464 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:00.339464 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:00.339464 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:00.340934 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:34:00.340934 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:34:00.340934 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:34:00.340934 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:34:00.340934 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:34:00.340934 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:34:00.340934 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:34:00.340934 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:34:00.351394 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 445/899] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:34:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:34:00.354361 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c53bf-f9fa-41ca-ba02-2882e430187d None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:00.382492 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4ac38-e11e-4253-a3b5-3c1ebe670542 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:00.425726 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c56a8-8762-441a-a0f5-d50ae80463eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:00.426324 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c56a8-8762-441a-a0f5-d50ae80463eb None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:00.426603 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c56a8-8762-441a-a0f5-d50ae80463eb None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:00.426945 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c56a8-8762-441a-a0f5-d50ae80463eb None admin] PATH_INFO: `/v3/services` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:00.427608 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2c56a8-8762-441a-a0f5-d50ae80463eb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:00.427919 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2c56a8-8762-441a-a0f5-d50ae80463eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:00.429353 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2c56a8-8762-441a-a0f5-d50ae80463eb None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:00.442886 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 455/900] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Sep 22 02:34:00 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:00.442886 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2f541-8ed3-40bc-ae9c-c06dced8c8d2 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:00.442886 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2f541-8ed3-40bc-ae9c-c06dced8c8d2 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:00.442886 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2f541-8ed3-40bc-ae9c-c06dced8c8d2 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:00.443822 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 456/901] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:34:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:34:00.448301 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb2f541-8ed3-40bc-ae9c-c06dced8c8d2 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:00.471793 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:00.526059 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:00.553364 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c53bf-f9fa-41ca-ba02-2882e430187d None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:00.604915 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb2f541-8ed3-40bc-ae9c-c06dced8c8d2 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:00.608710 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f89c0-210c-443b-b4c2-c81c6a6d2533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:00.609764 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f89c0-210c-443b-b4c2-c81c6a6d2533 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:00.610255 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f89c0-210c-443b-b4c2-c81c6a6d2533 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:00.610744 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f89c0-210c-443b-b4c2-c81c6a6d2533 None admin] PATH_INFO: `/v3/projects` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:00.612080 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f89c0-210c-443b-b4c2-c81c6a6d2533 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:00.612621 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f89c0-210c-443b-b4c2-c81c6a6d2533 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:00.615279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f89c0-210c-443b-b4c2-c81c6a6d2533 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:00.645901 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 446/902] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Sep 22 02:34:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:34:00.645901 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde18245-4c3c-4c64-9a85-e8c344747d27 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:00.645901 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde18245-4c3c-4c64-9a85-e8c344747d27 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:00.645901 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde18245-4c3c-4c64-9a85-e8c344747d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:00.659964 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-dde18245-4c3c-4c64-9a85-e8c344747d27 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:34:00.661395 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-dde18245-4c3c-4c64-9a85-e8c344747d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:34:00.686164 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dc9045-ec08-484d-a20e-70bbef1bd60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:00.687182 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc9045-ec08-484d-a20e-70bbef1bd60a None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:00.687682 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc9045-ec08-484d-a20e-70bbef1bd60a None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:00.688235 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc9045-ec08-484d-a20e-70bbef1bd60a None admin] PATH_INFO: `/v3/endpoints` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:00.689769 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dc9045-ec08-484d-a20e-70bbef1bd60a None admin] RBAC: Adding query filter params (interface=public, service_id=16794e858ff14e58afaf32acf8ff6f0f, region_id=RegionOne) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:00.690056 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dc9045-ec08-484d-a20e-70bbef1bd60a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:00.693239 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dc9045-ec08-484d-a20e-70bbef1bd60a None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:00.710875 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 457/903] 10.222.0.20 () {56 vars in 1251 bytes} [Thu Sep 22 02:34:00 2022] GET /identity/v3/endpoints?service_id=16794e858ff14e58afaf32acf8ff6f0f&interface=public®ion_id=RegionOne => generated 194 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:34:00.710875 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dc9045-ec08-484d-a20e-70bbef1bd60a None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:00.757288 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 447/904] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:34:00 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:34:00.767816 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde18245-4c3c-4c64-9a85-e8c344747d27 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:00.887882 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:00.927141 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dc9045-ec08-484d-a20e-70bbef1bd60a None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:00.928848 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:00.984719 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde18245-4c3c-4c64-9a85-e8c344747d27 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:01.001433 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d7ece-86d9-4136-9f45-363df7d042aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:01.003053 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d7ece-86d9-4136-9f45-363df7d042aa None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:01.003800 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d7ece-86d9-4136-9f45-363df7d042aa None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:01.005758 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d7ece-86d9-4136-9f45-363df7d042aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:01.007510 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428d7ece-86d9-4136-9f45-363df7d042aa None admin] RBAC: Adding query filter params (role.id=daf8ca6bfd534c00806f8cdfeb3bac5d, scope.project.id=2a792f8dc45145d687ddd58c94d44b5e, user.id=9821de2a961f4ddab2fca8e45b74e3db) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:01.008487 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428d7ece-86d9-4136-9f45-363df7d042aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:01.011523 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428d7ece-86d9-4136-9f45-363df7d042aa None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:01.027773 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 458/905] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:34:00 2022] GET /identity/v3/role_assignments?user.id=9821de2a961f4ddab2fca8e45b74e3db&scope.project.id=2a792f8dc45145d687ddd58c94d44b5e&role.id=daf8ca6bfd534c00806f8cdfeb3bac5d => generated 259 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:01.075585 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc1b06d-599c-417d-a124-1bd8c086c4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:01.077326 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1b06d-599c-417d-a124-1bd8c086c4d5 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:01.078118 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1b06d-599c-417d-a124-1bd8c086c4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:01.078978 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1b06d-599c-417d-a124-1bd8c086c4d5 None admin] PATH_INFO: `/v3/services` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:01.080656 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc1b06d-599c-417d-a124-1bd8c086c4d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:01.083846 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc1b06d-599c-417d-a124-1bd8c086c4d5 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:01.096763 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 448/906] 10.222.0.20 () {56 vars in 1087 bytes} [Thu Sep 22 02:34:00 2022] GET /identity/v3/services => generated 1268 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 02:34:01.108098 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2d174-77a0-472a-9a98-625fc85caee7 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:01.108505 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2d174-77a0-472a-9a98-625fc85caee7 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:01.108677 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2d174-77a0-472a-9a98-625fc85caee7 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:01.114995 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 459/907] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:34:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:34:01.117322 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d658870-3b3a-4adb-b12f-7923cd4db5f8 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:01.117588 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d658870-3b3a-4adb-b12f-7923cd4db5f8 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:01.117930 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d658870-3b3a-4adb-b12f-7923cd4db5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:01.148114 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-5d658870-3b3a-4adb-b12f-7923cd4db5f8 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:34:01.151058 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-5d658870-3b3a-4adb-b12f-7923cd4db5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:34:01.301923 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 449/908] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:34:01 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 02:34:01.325668 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad2d174-77a0-472a-9a98-625fc85caee7 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:01.522436 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad2d174-77a0-472a-9a98-625fc85caee7 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:01.579753 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d658870-3b3a-4adb-b12f-7923cd4db5f8 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:01.585638 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a32c8a4-112c-4910-9394-3ceaa868960b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:01.586678 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a32c8a4-112c-4910-9394-3ceaa868960b None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:01.587247 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a32c8a4-112c-4910-9394-3ceaa868960b None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:01.587790 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a32c8a4-112c-4910-9394-3ceaa868960b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:01.588951 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a32c8a4-112c-4910-9394-3ceaa868960b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:01.591701 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a32c8a4-112c-4910-9394-3ceaa868960b None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:01.611285 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-6a32c8a4-112c-4910-9394-3ceaa868960b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 02:34:01.611285 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:01.611285 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 02:34:01.611285 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 02:34:01.611285 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 02:34:01.611285 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 02:34:01.611285 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 02:34:01.611285 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:34:01.611285 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 02:34:01.611285 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:01.611285 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 02:34:01.611285 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:01.611285 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:01.611285 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:01.611285 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:01.611285 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:01.611285 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:01.612405 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:01.612405 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:01.612405 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:01.612405 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:01.612405 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:01.612405 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:01.612405 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 02:34:01.612405 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 02:34:01.612405 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 02:34:01.612405 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 02:34:01.612405 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:01.612405 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:01.612405 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:01.612405 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:01.612405 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:01.612405 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:01.613262 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:01.613262 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:01.613262 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:01.613262 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:01.613262 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:01.613262 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:01.613262 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:01.613262 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:01.613262 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:01.613262 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:01.613262 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 02:34:01.613262 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:34:01.613262 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 02:34:01.613262 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:01.614951 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 460/909] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Sep 22 02:34:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:34:01.621757 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a32c8a4-112c-4910-9394-3ceaa868960b None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:01.640532 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:01.709131 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:01.771319 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d658870-3b3a-4adb-b12f-7923cd4db5f8 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:01.824066 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a32c8a4-112c-4910-9394-3ceaa868960b None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:01.836230 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b62b95-e7fc-4eff-812c-1469f9df28c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:01.837405 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b62b95-e7fc-4eff-812c-1469f9df28c8 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:01.837907 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b62b95-e7fc-4eff-812c-1469f9df28c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:01.838472 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b62b95-e7fc-4eff-812c-1469f9df28c8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:01.839612 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b62b95-e7fc-4eff-812c-1469f9df28c8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:01.842343 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b62b95-e7fc-4eff-812c-1469f9df28c8 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:01.860358 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-38b62b95-e7fc-4eff-812c-1469f9df28c8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 02:34:01.860358 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:01.860358 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 02:34:01.860358 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 02:34:01.860358 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 02:34:01.860358 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 02:34:01.860358 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 02:34:01.860358 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:34:01.860358 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 02:34:01.860358 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:34:01.860358 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 02:34:01.860358 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:34:01.860358 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:01.860358 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:01.860358 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:01.860358 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:01.862210 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:01.862210 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:01.862210 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:01.862210 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:01.862210 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:01.862210 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:01.862210 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:01.862210 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 02:34:01.862210 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 02:34:01.862210 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 02:34:01.862210 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 02:34:01.862210 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:01.862210 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:01.862210 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:01.862210 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:01.862210 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:01.862210 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:01.865530 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:01.865530 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:01.865530 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:01.865530 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:01.865530 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:01.865530 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:01.865530 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:01.865530 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:01.865530 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:01.865530 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:01.865530 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 02:34:01.865530 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:34:01.865530 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 02:34:01.865530 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:34:01.865530 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 450/910] 10.222.0.20 () {56 vars in 1132 bytes} [Thu Sep 22 02:34:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:34:01.874869 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b62b95-e7fc-4eff-812c-1469f9df28c8 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:01.892918 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47de943-45bd-41a6-aa45-4de9cd59dfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:01.894076 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47de943-45bd-41a6-aa45-4de9cd59dfef None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:01.894597 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47de943-45bd-41a6-aa45-4de9cd59dfef None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:01.895128 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47de943-45bd-41a6-aa45-4de9cd59dfef None admin] PATH_INFO: `/v3/services` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:01.896293 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47de943-45bd-41a6-aa45-4de9cd59dfef None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:01.896830 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47de943-45bd-41a6-aa45-4de9cd59dfef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:01.899698 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47de943-45bd-41a6-aa45-4de9cd59dfef None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:01.909273 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 461/911] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Sep 22 02:34:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:34:01.915545 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47de943-45bd-41a6-aa45-4de9cd59dfef None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:02.000387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:02.049822 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47de943-45bd-41a6-aa45-4de9cd59dfef None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:02.069473 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b62b95-e7fc-4eff-812c-1469f9df28c8 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:02.112633 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:02.116271 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ae74c-5315-41c0-85e6-c3ca8783cc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:02.117604 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ae74c-5315-41c0-85e6-c3ca8783cc41 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:02.118201 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ae74c-5315-41c0-85e6-c3ca8783cc41 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:02.118772 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ae74c-5315-41c0-85e6-c3ca8783cc41 None admin] PATH_INFO: `/v3/services` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:02.120079 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ae74c-5315-41c0-85e6-c3ca8783cc41 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:02.120922 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ae74c-5315-41c0-85e6-c3ca8783cc41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:02.122223 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e539614-9b99-463f-a635-4c601c330a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:02.123089 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e539614-9b99-463f-a635-4c601c330a54 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:02.123547 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e539614-9b99-463f-a635-4c601c330a54 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:02.123926 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ae74c-5315-41c0-85e6-c3ca8783cc41 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:02.124199 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e539614-9b99-463f-a635-4c601c330a54 None admin] PATH_INFO: `/v3/services` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:02.125114 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e539614-9b99-463f-a635-4c601c330a54 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:02.125624 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e539614-9b99-463f-a635-4c601c330a54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:02.127594 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e539614-9b99-463f-a635-4c601c330a54 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:02.143247 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 462/912] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Sep 22 02:34:01 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:02.143467 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 451/913] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Sep 22 02:34:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:02.152870 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ae74c-5315-41c0-85e6-c3ca8783cc41 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:02.343001 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ae74c-5315-41c0-85e6-c3ca8783cc41 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:02.389858 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac839a2b-df40-4838-bb12-c688b05cc1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:02.390855 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac839a2b-df40-4838-bb12-c688b05cc1ff None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:02.391367 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac839a2b-df40-4838-bb12-c688b05cc1ff None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:02.391812 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac839a2b-df40-4838-bb12-c688b05cc1ff None admin] PATH_INFO: `/v3/services` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:02.392939 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac839a2b-df40-4838-bb12-c688b05cc1ff None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:02.393490 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac839a2b-df40-4838-bb12-c688b05cc1ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:02.396652 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac839a2b-df40-4838-bb12-c688b05cc1ff None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:02.400960 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:02.405890 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 463/914] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Sep 22 02:34:02 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:02.415738 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e539614-9b99-463f-a635-4c601c330a54 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:02.609765 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e539614-9b99-463f-a635-4c601c330a54 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:02.674813 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05358563-5cc2-4db5-b3af-635fa666c2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:02.675925 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05358563-5cc2-4db5-b3af-635fa666c2bf None admin] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:02.676421 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05358563-5cc2-4db5-b3af-635fa666c2bf None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:02.676905 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05358563-5cc2-4db5-b3af-635fa666c2bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:02.678127 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05358563-5cc2-4db5-b3af-635fa666c2bf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:02.681037 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05358563-5cc2-4db5-b3af-635fa666c2bf None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:02.719619 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 452/915] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Sep 22 02:34:02 2022] POST /identity/v3/endpoints => generated 347 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:34:03.892498 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96e754-e2d9-41d1-9c7a-84587bf26375 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:03.893059 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96e754-e2d9-41d1-9c7a-84587bf26375 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:03.893059 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96e754-e2d9-41d1-9c7a-84587bf26375 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:03.894847 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 464/916] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:34:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:34:03.900959 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50115f-74f5-4aa4-9612-4b759c96bf79 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:03.901936 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50115f-74f5-4aa4-9612-4b759c96bf79 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:03.902408 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50115f-74f5-4aa4-9612-4b759c96bf79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:03.922122 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-4f50115f-74f5-4aa4-9612-4b759c96bf79 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:34:03.924000 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-4f50115f-74f5-4aa4-9612-4b759c96bf79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:34:03.994323 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:03.998884 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2709874b-1636-4393-b9d6-d6b779842689 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:03.998884 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2709874b-1636-4393-b9d6-d6b779842689 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:03.999169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2709874b-1636-4393-b9d6-d6b779842689 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:04.001764 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 465/917] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:34:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:34:04.007165 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad1fab-0492-4ea7-b31d-d6cc86a6f6d6 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:04.007590 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad1fab-0492-4ea7-b31d-d6cc86a6f6d6 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:04.007960 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad1fab-0492-4ea7-b31d-d6cc86a6f6d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:04.030310 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-6fad1fab-0492-4ea7-b31d-d6cc86a6f6d6 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:34:04.031556 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-6fad1fab-0492-4ea7-b31d-d6cc86a6f6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:34:04.047710 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 453/918] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:34:03 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:34:04.119063 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 466/919] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:34:04 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:34:04.396103 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f50115f-74f5-4aa4-9612-4b759c96bf79 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:04.521041 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad1fab-0492-4ea7-b31d-d6cc86a6f6d6 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:04.585645 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f50115f-74f5-4aa4-9612-4b759c96bf79 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:04.656009 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c9a269-0c10-42e3-8975-c0c7ceb5e9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:04.657422 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c9a269-0c10-42e3-8975-c0c7ceb5e9d5 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:04.658137 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c9a269-0c10-42e3-8975-c0c7ceb5e9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:04.658891 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c9a269-0c10-42e3-8975-c0c7ceb5e9d5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:04.685168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c9a269-0c10-42e3-8975-c0c7ceb5e9d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:34:04.695238 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-04c9a269-0c10-42e3-8975-c0c7ceb5e9d5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 02:34:04.695238 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:04.695238 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:04.695238 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:04.695238 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:04.695238 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:04.695238 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:04.695238 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:04.695238 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:04.695238 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:04.695238 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:04.695238 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:04.695238 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:34:04.695238 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:34:04.695238 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:34:04.695238 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:34:04.697279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:34:04.697279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:34:04.697279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:34:04.697279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:34:04.697279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:04.697279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:04.697279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:04.697279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:04.697279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:04.697279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:04.697279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:04.697279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:04.697279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:04.697279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:04.697279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:04.697279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:04.699252 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:04.699252 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:04.699252 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:04.699252 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:04.699252 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:34:04.699252 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:34:04.699252 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:34:04.699252 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:34:04.699252 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:34:04.699252 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:34:04.699252 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 02:34:04.699252 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:34:04.703094 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 454/920] 10.222.0.20 () {56 vars in 1120 bytes} [Thu Sep 22 02:34:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:34:04.713237 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c9a269-0c10-42e3-8975-c0c7ceb5e9d5 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:04.716809 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad1fab-0492-4ea7-b31d-d6cc86a6f6d6 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:04.753401 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:04.761424 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3523954-83e4-464a-a738-785a8a53d790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:04.762106 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3523954-83e4-464a-a738-785a8a53d790 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:04.762394 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3523954-83e4-464a-a738-785a8a53d790 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:04.762679 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3523954-83e4-464a-a738-785a8a53d790 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:04.763343 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3523954-83e4-464a-a738-785a8a53d790 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:04.764788 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3523954-83e4-464a-a738-785a8a53d790 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:04.766172 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:04.776800 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-d3523954-83e4-464a-a738-785a8a53d790 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 02:34:04.776800 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:04.776800 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 02:34:04.776800 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 02:34:04.776800 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 02:34:04.776800 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 02:34:04.776800 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 02:34:04.776800 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:34:04.776800 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 02:34:04.776800 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:04.776800 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 02:34:04.776800 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:04.776800 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:04.776800 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:04.776800 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:04.776800 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:04.776800 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:04.778008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:04.778008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:04.778008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:04.778008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:04.778008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:04.778008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:04.778008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 02:34:04.778008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 02:34:04.778008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 02:34:04.778008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 02:34:04.778008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:04.778008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:04.778008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:04.778008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:04.778008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:04.778008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:04.778928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:04.778928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:04.778928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:04.778928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:04.778928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:04.778928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:04.778928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:04.778928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:04.778928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:04.778928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:04.778928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 02:34:04.778928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:34:04.778928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 02:34:04.778928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:04.783275 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 467/921] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Sep 22 02:34:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:34:04.785067 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40736df4-75b2-400f-aae1-e3621d951a6b None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:04.785896 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40736df4-75b2-400f-aae1-e3621d951a6b None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:04.786771 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40736df4-75b2-400f-aae1-e3621d951a6b None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:04.793056 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 468/922] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:34:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:34:04.793056 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40736df4-75b2-400f-aae1-e3621d951a6b None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:04.877912 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c9a269-0c10-42e3-8975-c0c7ceb5e9d5 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:04.920339 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40736df4-75b2-400f-aae1-e3621d951a6b None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:04.923887 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c975fef-0e4b-4274-b509-6ebc231ea098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:04.925077 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c975fef-0e4b-4274-b509-6ebc231ea098 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:04.925524 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c975fef-0e4b-4274-b509-6ebc231ea098 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:04.925950 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c975fef-0e4b-4274-b509-6ebc231ea098 None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:04.926782 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c975fef-0e4b-4274-b509-6ebc231ea098 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:04.927282 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c975fef-0e4b-4274-b509-6ebc231ea098 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:04.929436 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c975fef-0e4b-4274-b509-6ebc231ea098 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:04.942297 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 455/923] 10.222.0.20 () {56 vars in 1115 bytes} [Thu Sep 22 02:34:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:04.943970 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0098c-68e8-4b30-b771-ca654c5a87e1 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:04.944942 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0098c-68e8-4b30-b771-ca654c5a87e1 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:04.945362 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0098c-68e8-4b30-b771-ca654c5a87e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:04.959559 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3eb50-96f8-4e65-9a38-0b300d98e2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:04.960184 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3eb50-96f8-4e65-9a38-0b300d98e2e3 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:04.960466 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3eb50-96f8-4e65-9a38-0b300d98e2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:04.960741 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3eb50-96f8-4e65-9a38-0b300d98e2e3 None admin] PATH_INFO: `/v3/services` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:04.961382 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a3eb50-96f8-4e65-9a38-0b300d98e2e3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:04.961691 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a3eb50-96f8-4e65-9a38-0b300d98e2e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:04.963113 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a3eb50-96f8-4e65-9a38-0b300d98e2e3 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:04.972963 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 469/924] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Sep 22 02:34:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:04.972963 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3eb50-96f8-4e65-9a38-0b300d98e2e3 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:04.973248 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-5aa0098c-68e8-4b30-b771-ca654c5a87e1 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:34:04.975232 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-5aa0098c-68e8-4b30-b771-ca654c5a87e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:34:05.047699 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:05.079183 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 456/925] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:34:04 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:34:05.080530 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa0098c-68e8-4b30-b771-ca654c5a87e1 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:05.123463 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3eb50-96f8-4e65-9a38-0b300d98e2e3 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:05.140379 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:05.177829 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce8003c-c77f-4ed5-87a0-920d50a9d57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:05.179053 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8003c-c77f-4ed5-87a0-920d50a9d57a None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:05.179441 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8003c-c77f-4ed5-87a0-920d50a9d57a None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:05.179856 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8003c-c77f-4ed5-87a0-920d50a9d57a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:05.197162 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce8003c-c77f-4ed5-87a0-920d50a9d57a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:05.198956 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce8003c-c77f-4ed5-87a0-920d50a9d57a None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:05.217097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-fce8003c-c77f-4ed5-87a0-920d50a9d57a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:34:05.217097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:05.217097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:05.217097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:05.217097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:05.217097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:05.217097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:05.217097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:05.217097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:05.217097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:05.217097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:05.217097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:05.217097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:34:05.217097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:34:05.217097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:34:05.217097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:34:05.219089 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:05.219089 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:05.219089 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:05.219089 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:05.219089 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:05.219089 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:05.219089 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:05.219089 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:05.219089 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:05.219089 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:05.219089 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:05.219089 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:05.219089 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:05.219089 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:05.219089 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:05.219089 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:05.219089 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:34:05.220763 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:34:05.220763 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:34:05.220763 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:05.220763 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 470/926] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:34:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:34:05.230463 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce8003c-c77f-4ed5-87a0-920d50a9d57a None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:05.236492 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa0098c-68e8-4b30-b771-ca654c5a87e1 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:05.282657 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda4b48-cc9c-4ed6-811a-a8e1b41e10ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:05.283333 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda4b48-cc9c-4ed6-811a-a8e1b41e10ed None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:05.283652 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda4b48-cc9c-4ed6-811a-a8e1b41e10ed None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:05.283897 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda4b48-cc9c-4ed6-811a-a8e1b41e10ed None admin] PATH_INFO: `/v3/services` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:05.284554 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbda4b48-cc9c-4ed6-811a-a8e1b41e10ed None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:05.284863 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbda4b48-cc9c-4ed6-811a-a8e1b41e10ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:05.286346 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbda4b48-cc9c-4ed6-811a-a8e1b41e10ed None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:05.295145 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 457/927] 10.222.0.20 () {56 vars in 1114 bytes} [Thu Sep 22 02:34:05 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:05.298655 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda4b48-cc9c-4ed6-811a-a8e1b41e10ed None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:05.406784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:05.414337 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda4b48-cc9c-4ed6-811a-a8e1b41e10ed None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:05.424461 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce8003c-c77f-4ed5-87a0-920d50a9d57a None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:05.439067 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:05.457163 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a6fcf-4df5-49c8-8a6a-74dd9add3317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:05.457863 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a6fcf-4df5-49c8-8a6a-74dd9add3317 None admin] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:05.458154 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a6fcf-4df5-49c8-8a6a-74dd9add3317 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:05.458438 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a6fcf-4df5-49c8-8a6a-74dd9add3317 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:05.459193 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91a6fcf-4df5-49c8-8a6a-74dd9add3317 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:05.460632 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91a6fcf-4df5-49c8-8a6a-74dd9add3317 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:05.494872 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 458/928] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Sep 22 02:34:05 2022] POST /identity/v3/endpoints => generated 346 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 02:34:05.494872 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a6fcf-4df5-49c8-8a6a-74dd9add3317 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:05.506413 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4e734f-2044-4c35-8c70-b2304ad9d90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:05.507804 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e734f-2044-4c35-8c70-b2304ad9d90a None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:05.508550 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e734f-2044-4c35-8c70-b2304ad9d90a None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:05.509056 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e734f-2044-4c35-8c70-b2304ad9d90a None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:05.510421 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4e734f-2044-4c35-8c70-b2304ad9d90a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:05.511021 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4e734f-2044-4c35-8c70-b2304ad9d90a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:05.513766 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4e734f-2044-4c35-8c70-b2304ad9d90a None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:05.533363 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 471/929] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:34:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:34:05.536530 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18dabb-7f78-4853-8ace-8473e34fb619 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:05.537186 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18dabb-7f78-4853-8ace-8473e34fb619 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:05.537731 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18dabb-7f78-4853-8ace-8473e34fb619 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:05.540618 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 472/930] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:34:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:34:05.544229 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18dabb-7f78-4853-8ace-8473e34fb619 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:05.692579 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a6fcf-4df5-49c8-8a6a-74dd9add3317 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:05.695962 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18dabb-7f78-4853-8ace-8473e34fb619 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:05.738312 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a268768-4f17-46c5-8635-727e7e50f0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:05.739104 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a268768-4f17-46c5-8635-727e7e50f0ea None admin] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:05.739328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a268768-4f17-46c5-8635-727e7e50f0ea None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:05.739631 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a268768-4f17-46c5-8635-727e7e50f0ea None admin] PATH_INFO: `/v3/services` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:05.740298 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a268768-4f17-46c5-8635-727e7e50f0ea None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:05.741700 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a268768-4f17-46c5-8635-727e7e50f0ea None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:05.748315 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e8b30-e026-4703-bb64-e2f26f7daee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:05.749141 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e8b30-e026-4703-bb64-e2f26f7daee6 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:05.749664 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e8b30-e026-4703-bb64-e2f26f7daee6 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:05.750037 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e8b30-e026-4703-bb64-e2f26f7daee6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:05.761228 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 459/931] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Sep 22 02:34:05 2022] POST /identity/v3/services => generated 241 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:34:05.761489 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444e8b30-e026-4703-bb64-e2f26f7daee6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:05.762803 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444e8b30-e026-4703-bb64-e2f26f7daee6 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:05.763275 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0728af1-612f-4328-b8a6-d3dfb3c8d125 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:05.763587 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0728af1-612f-4328-b8a6-d3dfb3c8d125 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:05.763895 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0728af1-612f-4328-b8a6-d3dfb3c8d125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:05.780077 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-444e8b30-e026-4703-bb64-e2f26f7daee6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 02:34:05.780077 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:05.780077 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:05.780077 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:05.780077 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:05.780077 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:05.780077 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:05.780077 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:05.780077 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:05.780077 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:05.780077 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:05.780077 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:05.780077 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:34:05.780077 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:34:05.780077 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:34:05.780077 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:34:05.781108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:05.781108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:05.781108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:34:05.781108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:34:05.781108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:34:05.781108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:34:05.781108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:05.781108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:05.781108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:05.781108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:05.781108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:05.781108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:05.781108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:05.781108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:05.781108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:05.781108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:05.781108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:05.781108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:05.782079 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:05.782079 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:05.782079 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:34:05.782079 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:34:05.782079 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:34:05.782079 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:34:05.782079 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:34:05.782079 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:34:05.782079 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 02:34:05.782079 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:05.784869 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 473/932] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Sep 22 02:34:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:34:05.788188 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-c0728af1-612f-4328-b8a6-d3dfb3c8d125 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:34:05.790924 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e8b30-e026-4703-bb64-e2f26f7daee6 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:05.791098 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-c0728af1-612f-4328-b8a6-d3dfb3c8d125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:34:05.795610 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:05.837718 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:05.921196 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 460/933] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:34:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:34:05.944195 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e8b30-e026-4703-bb64-e2f26f7daee6 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:05.986411 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4859d9d0-8bf4-4ee8-8fd5-6cd2200f4c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:05.987198 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859d9d0-8bf4-4ee8-8fd5-6cd2200f4c0c None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:05.987571 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859d9d0-8bf4-4ee8-8fd5-6cd2200f4c0c None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:05.987930 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859d9d0-8bf4-4ee8-8fd5-6cd2200f4c0c None admin] PATH_INFO: `/v3/users` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:05.989869 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4859d9d0-8bf4-4ee8-8fd5-6cd2200f4c0c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:05.990398 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4859d9d0-8bf4-4ee8-8fd5-6cd2200f4c0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:05.992362 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4859d9d0-8bf4-4ee8-8fd5-6cd2200f4c0c None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:06.024556 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 474/934] 10.222.0.20 () {56 vars in 1149 bytes} [Thu Sep 22 02:34:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:06.029705 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0728af1-612f-4328-b8a6-d3dfb3c8d125 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:06.146047 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0728af1-612f-4328-b8a6-d3dfb3c8d125 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:06.187276 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce4088c-b73a-4e2a-9f91-b67e3a2983ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:06.187990 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce4088c-b73a-4e2a-9f91-b67e3a2983ba None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:06.188272 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce4088c-b73a-4e2a-9f91-b67e3a2983ba None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:06.188550 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce4088c-b73a-4e2a-9f91-b67e3a2983ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:06.192537 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:06.202251 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce4088c-b73a-4e2a-9f91-b67e3a2983ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:06.203739 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce4088c-b73a-4e2a-9f91-b67e3a2983ba None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:06.218050 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4859d9d0-8bf4-4ee8-8fd5-6cd2200f4c0c None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:06.223032 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-9ce4088c-b73a-4e2a-9f91-b67e3a2983ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:34:06.223032 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:06.223032 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:06.223032 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:06.223032 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:06.223032 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:06.223032 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:06.223032 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:06.223032 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:06.223032 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:06.223032 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:06.223032 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:06.223032 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:34:06.223032 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:34:06.223032 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:34:06.223032 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:34:06.224087 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:06.224087 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:06.224087 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:06.224087 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:06.224087 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:06.224087 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:06.224087 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:06.224087 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:06.224087 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:06.224087 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:06.224087 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:06.224087 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:06.224087 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:06.224087 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:06.224087 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:06.224087 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:06.224087 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:34:06.224998 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:34:06.224998 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:34:06.224998 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:34:06.230929 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 461/935] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:34:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:34:06.235471 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce4088c-b73a-4e2a-9f91-b67e3a2983ba None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:06.347635 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:06.358983 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4859d9d0-8bf4-4ee8-8fd5-6cd2200f4c0c None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:06.398573 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce4088c-b73a-4e2a-9f91-b67e3a2983ba None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:06.407154 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66654f-6f41-415b-9e39-a74fa2eed055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:06.407918 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66654f-6f41-415b-9e39-a74fa2eed055 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:06.408205 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66654f-6f41-415b-9e39-a74fa2eed055 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:06.408479 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66654f-6f41-415b-9e39-a74fa2eed055 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:06.409110 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a66654f-6f41-415b-9e39-a74fa2eed055 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:06.410536 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a66654f-6f41-415b-9e39-a74fa2eed055 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:06.421347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-6a66654f-6f41-415b-9e39-a74fa2eed055 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 02:34:06.421347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:06.421347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 02:34:06.421347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 02:34:06.421347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 02:34:06.421347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 02:34:06.421347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 02:34:06.421347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:34:06.421347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 02:34:06.421347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:06.421347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 02:34:06.421347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:06.421347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:06.421347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:06.421347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:06.421347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:06.421347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:06.423321 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:06.423321 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:06.423321 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:06.423321 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:06.423321 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:06.423321 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:06.423321 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 02:34:06.423321 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 02:34:06.423321 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 02:34:06.423321 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 02:34:06.423321 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:06.423321 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:06.423321 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:06.423321 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:06.423321 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:06.423321 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:06.424962 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:06.424962 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:06.424962 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:06.424962 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:06.424962 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:06.424962 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:06.424962 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:06.424962 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:06.424962 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:06.424962 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:06.424962 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 02:34:06.424962 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:34:06.424962 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 02:34:06.424962 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:06.428323 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 475/936] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:34:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:34:06.429775 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66654f-6f41-415b-9e39-a74fa2eed055 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:06.496287 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad5473b-6529-471a-ad15-f80895ddfb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:06.497839 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5473b-6529-471a-ad15-f80895ddfb22 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:06.498610 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5473b-6529-471a-ad15-f80895ddfb22 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:06.499506 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5473b-6529-471a-ad15-f80895ddfb22 None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:06.499965 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad5473b-6529-471a-ad15-f80895ddfb22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:06.500277 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad5473b-6529-471a-ad15-f80895ddfb22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:06.501721 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad5473b-6529-471a-ad15-f80895ddfb22 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:06.521471 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 462/937] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:34:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:06.523228 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad5473b-6529-471a-ad15-f80895ddfb22 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:06.594636 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66654f-6f41-415b-9e39-a74fa2eed055 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:06.607551 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:06.649838 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2197bc3-10c0-4546-b987-a490d73f6b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:06.651483 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2197bc3-10c0-4546-b987-a490d73f6b17 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:06.652468 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2197bc3-10c0-4546-b987-a490d73f6b17 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:06.654394 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2197bc3-10c0-4546-b987-a490d73f6b17 None admin] PATH_INFO: `/v3/services` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:06.655757 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2197bc3-10c0-4546-b987-a490d73f6b17 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:06.656583 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2197bc3-10c0-4546-b987-a490d73f6b17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:06.660259 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2197bc3-10c0-4546-b987-a490d73f6b17 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:06.678101 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 476/938] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Sep 22 02:34:06 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:06.679263 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2197bc3-10c0-4546-b987-a490d73f6b17 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:06.698022 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:06.735861 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad5473b-6529-471a-ad15-f80895ddfb22 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:06.778132 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0803e294-10b8-4357-b2f2-d3186f3c1222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:06.778687 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803e294-10b8-4357-b2f2-d3186f3c1222 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:06.778995 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803e294-10b8-4357-b2f2-d3186f3c1222 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:06.779271 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803e294-10b8-4357-b2f2-d3186f3c1222 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:06.787495 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0803e294-10b8-4357-b2f2-d3186f3c1222 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:06.788985 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0803e294-10b8-4357-b2f2-d3186f3c1222 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:06.803328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-0803e294-10b8-4357-b2f2-d3186f3c1222 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:34:06.803328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:06.803328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:06.803328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:06.803328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:06.803328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:06.803328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:06.803328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:06.803328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:06.803328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:06.803328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:06.803328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:06.803328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:34:06.803328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:34:06.803328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:34:06.805119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:34:06.805119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:06.805119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:06.805119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:06.805119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:06.805119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:06.805119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:06.805119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:06.805119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:06.805119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:06.805119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:06.805119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:06.805119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:06.805119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:06.805119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:06.805119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:06.805119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:06.807641 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:34:06.807641 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:34:06.807641 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:34:06.807641 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:34:06.807641 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:34:06.807641 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:34:06.807641 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:34:06.807641 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:34:06.809608 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 463/939] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:34:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:34:06.811969 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0803e294-10b8-4357-b2f2-d3186f3c1222 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:06.877710 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2197bc3-10c0-4546-b987-a490d73f6b17 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:06.956975 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a2a283-5811-4bf0-9929-21ceffc96d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:06.960185 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2a283-5811-4bf0-9929-21ceffc96d1b None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:06.960990 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2a283-5811-4bf0-9929-21ceffc96d1b None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:06.961428 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0803e294-10b8-4357-b2f2-d3186f3c1222 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:06.962091 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2a283-5811-4bf0-9929-21ceffc96d1b None admin] PATH_INFO: `/v3/services` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:06.965213 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a2a283-5811-4bf0-9929-21ceffc96d1b None admin] RBAC: Adding query filter params (type=compute) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:06.966086 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a2a283-5811-4bf0-9929-21ceffc96d1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:06.968991 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a2a283-5811-4bf0-9929-21ceffc96d1b None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:06.973526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:06.988859 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 477/940] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Sep 22 02:34:06 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:34:07.015599 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b6a0ab-5e1d-45e9-a2dd-505fad1983bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:07.016255 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6a0ab-5e1d-45e9-a2dd-505fad1983bf None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:07.016539 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6a0ab-5e1d-45e9-a2dd-505fad1983bf None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:07.016812 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6a0ab-5e1d-45e9-a2dd-505fad1983bf None admin] PATH_INFO: `/v3/projects` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:07.017553 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b6a0ab-5e1d-45e9-a2dd-505fad1983bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:07.017941 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b6a0ab-5e1d-45e9-a2dd-505fad1983bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:07.020175 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b6a0ab-5e1d-45e9-a2dd-505fad1983bf None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:07.042265 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 464/941] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Sep 22 02:34:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:07.044548 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a2a283-5811-4bf0-9929-21ceffc96d1b None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:07.143796 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:07.174181 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a2a283-5811-4bf0-9929-21ceffc96d1b None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:07.227130 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7ef4f-6b12-4dbb-a4a2-5950b675f444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:07.227866 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7ef4f-6b12-4dbb-a4a2-5950b675f444 None admin] REQUEST_METHOD: `PUT` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:07.228153 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7ef4f-6b12-4dbb-a4a2-5950b675f444 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:07.228437 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7ef4f-6b12-4dbb-a4a2-5950b675f444 None admin] PATH_INFO: `/v3/projects/2a792f8dc45145d687ddd58c94d44b5e/users/9821de2a961f4ddab2fca8e45b74e3db/roles/daf8ca6bfd534c00806f8cdfeb3bac5d` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:07.230913 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d7ef4f-6b12-4dbb-a4a2-5950b675f444 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a792f8dc45145d687ddd58c94d44b5e, user_id=9821de2a961f4ddab2fca8e45b74e3db, role_id=daf8ca6bfd534c00806f8cdfeb3bac5d)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:07.232410 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d7ef4f-6b12-4dbb-a4a2-5950b675f444 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:07.250112 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 478/942] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Sep 22 02:34:07 2022] PUT /identity/v3/projects/2a792f8dc45145d687ddd58c94d44b5e/users/9821de2a961f4ddab2fca8e45b74e3db/roles/daf8ca6bfd534c00806f8cdfeb3bac5d => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 02:34:08.062318 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f49556-93fb-4d3b-a737-0f4fcbcadc8a None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:08.063321 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f49556-93fb-4d3b-a737-0f4fcbcadc8a None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:08.063321 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f49556-93fb-4d3b-a737-0f4fcbcadc8a None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:08.064618 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 465/943] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:34:08.070503 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ce403-d59e-438c-8e02-018f9a44489b None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:08.071076 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ce403-d59e-438c-8e02-018f9a44489b None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:08.071076 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ce403-d59e-438c-8e02-018f9a44489b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:08.091442 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-a20ce403-d59e-438c-8e02-018f9a44489b None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:34:08.092757 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-a20ce403-d59e-438c-8e02-018f9a44489b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:34:08.188770 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 479/944] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:34:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:34:08.635673 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f49556-93fb-4d3b-a737-0f4fcbcadc8a None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:08.745144 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9e9e8-d74d-4dce-979e-aaa7a85adbdc None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:08.745518 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9e9e8-d74d-4dce-979e-aaa7a85adbdc None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:08.745697 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9e9e8-d74d-4dce-979e-aaa7a85adbdc None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:08.747323 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 480/945] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:34:08.752510 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65785b73-f647-4b33-99e9-166645889ee6 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:08.752800 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65785b73-f647-4b33-99e9-166645889ee6 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:08.753075 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65785b73-f647-4b33-99e9-166645889ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:08.762670 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f49556-93fb-4d3b-a737-0f4fcbcadc8a None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:08.773293 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-65785b73-f647-4b33-99e9-166645889ee6 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:34:08.774487 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-65785b73-f647-4b33-99e9-166645889ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:34:08.807589 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eea1dd-ba3a-46ca-91c3-f4fbbe46b662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:08.808782 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eea1dd-ba3a-46ca-91c3-f4fbbe46b662 None admin] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:08.809343 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eea1dd-ba3a-46ca-91c3-f4fbbe46b662 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:08.809833 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eea1dd-ba3a-46ca-91c3-f4fbbe46b662 None admin] PATH_INFO: `/v3/projects` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:08.810715 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eea1dd-ba3a-46ca-91c3-f4fbbe46b662 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:08.813494 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eea1dd-ba3a-46ca-91c3-f4fbbe46b662 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:08.817052 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: WARNING keystone.server.flask.common [None req-c2eea1dd-ba3a-46ca-91c3-f4fbbe46b662 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 22 02:34:08.841221 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.sql.core [None req-c2eea1dd-ba3a-46ca-91c3-f4fbbe46b662 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 22 02:34:08.841221 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 22 02:34:08.841221 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [parameters: {'id': '4486e67910fd4841a162cf839795ad1c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 22 02:34:08.841221 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88065) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 22 02:34:08.845336 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:08.853147 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-c2eea1dd-ba3a-46ca-91c3-f4fbbe46b662 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4486e67910fd4841a162cf839795ad1c.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4486e67910fd4841a162cf839795ad1c. Sep 22 02:34:08.853147 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:08.853147 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 22 02:34:08.853147 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 22 02:34:08.853147 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 22 02:34:08.853147 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 22 02:34:08.853147 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 22 02:34:08.853147 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application result = self._query(query) Sep 22 02:34:08.853147 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 22 02:34:08.853147 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application conn.query(q) Sep 22 02:34:08.853147 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 22 02:34:08.853147 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 22 02:34:08.853147 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 22 02:34:08.854511 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application result.read() Sep 22 02:34:08.854511 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 22 02:34:08.854511 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 22 02:34:08.854511 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 22 02:34:08.854511 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 22 02:34:08.854511 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 22 02:34:08.854511 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 22 02:34:08.854511 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 22 02:34:08.854511 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 22 02:34:08.854511 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 22 02:34:08.854511 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:34:08.854511 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 22 02:34:08.854511 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:34:08.854511 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:08.854511 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 22 02:34:08.854511 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 22 02:34:08.854511 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 22 02:34:08.854511 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 22 02:34:08.855614 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 22 02:34:08.855614 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application next(self.gen) Sep 22 02:34:08.855614 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 22 02:34:08.855614 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application yield resource Sep 22 02:34:08.855614 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 22 02:34:08.855614 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application next(self.gen) Sep 22 02:34:08.855614 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 22 02:34:08.855614 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self.session.rollback() Sep 22 02:34:08.855614 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 22 02:34:08.855614 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self.force_reraise() Sep 22 02:34:08.855614 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 22 02:34:08.855614 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise self.value Sep 22 02:34:08.855614 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 22 02:34:08.855614 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 22 02:34:08.855614 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 22 02:34:08.855614 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application session.commit() Sep 22 02:34:08.855614 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 22 02:34:08.856526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 22 02:34:08.856526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 22 02:34:08.856526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self._prepare_impl() Sep 22 02:34:08.856526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 22 02:34:08.856526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self.session.flush() Sep 22 02:34:08.856526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 22 02:34:08.856526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self._flush(objects) Sep 22 02:34:08.856526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 22 02:34:08.856526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 22 02:34:08.856526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 22 02:34:08.856526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application compat.raise_( Sep 22 02:34:08.856526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 22 02:34:08.856526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception Sep 22 02:34:08.856526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 22 02:34:08.856526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application flush_context.execute() Sep 22 02:34:08.856526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 22 02:34:08.856526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rec.execute(self) Sep 22 02:34:08.859787 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 22 02:34:08.859787 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 22 02:34:08.859787 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 22 02:34:08.859787 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 22 02:34:08.859787 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 22 02:34:08.859787 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 22 02:34:08.859787 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 22 02:34:08.859787 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 22 02:34:08.859787 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 22 02:34:08.859787 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 22 02:34:08.859787 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 22 02:34:08.859787 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 22 02:34:08.859787 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 22 02:34:08.859787 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 22 02:34:08.859787 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 22 02:34:08.860577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 22 02:34:08.860577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 22 02:34:08.860577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception Sep 22 02:34:08.860577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 22 02:34:08.860577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 22 02:34:08.860577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 22 02:34:08.860577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 22 02:34:08.860577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 22 02:34:08.860577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application result = self._query(query) Sep 22 02:34:08.860577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 22 02:34:08.860577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application conn.query(q) Sep 22 02:34:08.860577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 22 02:34:08.860577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 22 02:34:08.860577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 22 02:34:08.860577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application result.read() Sep 22 02:34:08.860577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 22 02:34:08.860577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 22 02:34:08.861509 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 22 02:34:08.861509 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 22 02:34:08.861509 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 22 02:34:08.861509 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 22 02:34:08.861509 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 22 02:34:08.861509 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 22 02:34:08.861509 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 22 02:34:08.861509 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 22 02:34:08.861509 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [parameters: {'id': '4486e67910fd4841a162cf839795ad1c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 22 02:34:08.861509 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 22 02:34:08.861509 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:34:08.861509 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 02:34:08.861509 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:34:08.861509 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:08.861509 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 22 02:34:08.862308 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 22 02:34:08.862308 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 22 02:34:08.862308 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 22 02:34:08.862308 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 22 02:34:08.862308 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:34:08.862308 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 02:34:08.862308 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:34:08.862308 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:08.862308 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:08.862308 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:08.862308 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:08.862308 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:08.862308 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:08.862308 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:08.862308 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:08.862308 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:08.862308 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:08.863294 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:08.863294 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 22 02:34:08.863294 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 22 02:34:08.863294 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:08.863294 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:08.863294 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 22 02:34:08.863294 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 22 02:34:08.863294 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4486e67910fd4841a162cf839795ad1c. Sep 22 02:34:08.863294 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:34:08.863294 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 466/946] 10.222.0.20 () {60 vars in 1140 bytes} [Thu Sep 22 02:34:08 2022] POST /identity/v3/projects => generated 262 bytes in 220 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:34:08.863968 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eea1dd-ba3a-46ca-91c3-f4fbbe46b662 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:08.866890 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 481/947] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:34:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:34:08.940412 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:09.039439 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eea1dd-ba3a-46ca-91c3-f4fbbe46b662 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:09.076350 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfe1a89-d40b-4a36-8dd7-d707a0daf94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:09.077238 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfe1a89-d40b-4a36-8dd7-d707a0daf94f None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:09.077238 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfe1a89-d40b-4a36-8dd7-d707a0daf94f None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:09.077710 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfe1a89-d40b-4a36-8dd7-d707a0daf94f None admin] PATH_INFO: `/v3/projects/service` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:09.085540 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfe1a89-d40b-4a36-8dd7-d707a0daf94f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:09.087308 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfe1a89-d40b-4a36-8dd7-d707a0daf94f None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:09.099856 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-fbfe1a89-d40b-4a36-8dd7-d707a0daf94f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:34:09.099856 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:09.099856 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:09.099856 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:09.099856 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:09.099856 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:09.099856 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:09.099856 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:09.099856 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:09.099856 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:09.099856 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:09.099856 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:09.099856 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:34:09.099856 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:34:09.099856 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:34:09.101413 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:34:09.101413 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:09.101413 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:09.101413 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:09.101413 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:09.101413 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:09.101413 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:09.101413 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:09.101413 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:09.101413 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:09.101413 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:09.101413 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:09.101413 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:09.101413 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:09.101413 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:09.101413 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:09.101413 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:09.103507 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:34:09.103507 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:34:09.103507 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:34:09.103507 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:34:09.103507 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:34:09.103507 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:34:09.103507 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:34:09.103507 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:34:09.103507 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 467/948] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:34:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:34:09.107471 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65785b73-f647-4b33-99e9-166645889ee6 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:09.225354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65785b73-f647-4b33-99e9-166645889ee6 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:09.272947 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c4adde-945e-48cc-adff-7ac4148c2903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:09.274019 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c4adde-945e-48cc-adff-7ac4148c2903 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:09.274523 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c4adde-945e-48cc-adff-7ac4148c2903 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:09.275272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c4adde-945e-48cc-adff-7ac4148c2903 None admin] PATH_INFO: `/v3/projects` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:09.276180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c4adde-945e-48cc-adff-7ac4148c2903 None admin] RBAC: Adding query filter params (name=service) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:09.276698 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c4adde-945e-48cc-adff-7ac4148c2903 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:09.278514 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c4adde-945e-48cc-adff-7ac4148c2903 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:09.301273 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 482/949] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Sep 22 02:34:09 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:09.322586 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfe1a89-d40b-4a36-8dd7-d707a0daf94f None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:09.459535 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:09.499494 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfe1a89-d40b-4a36-8dd7-d707a0daf94f None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:09.504182 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be5c40-2f1c-432e-9ea0-f0b7fadd8122 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:09.504839 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be5c40-2f1c-432e-9ea0-f0b7fadd8122 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:09.505386 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be5c40-2f1c-432e-9ea0-f0b7fadd8122 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:09.507365 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 483/950] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:34:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:34:09.517061 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3eca6d-342f-4da2-9b13-dfe95e6e96f2 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:09.517737 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3eca6d-342f-4da2-9b13-dfe95e6e96f2 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:09.518219 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3eca6d-342f-4da2-9b13-dfe95e6e96f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:09.540047 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-5a3eca6d-342f-4da2-9b13-dfe95e6e96f2 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:34:09.541721 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-5a3eca6d-342f-4da2-9b13-dfe95e6e96f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:34:09.575479 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8862d-e9ae-48ff-89e7-f84c7b984a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:09.576595 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8862d-e9ae-48ff-89e7-f84c7b984a08 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:09.577066 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8862d-e9ae-48ff-89e7-f84c7b984a08 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:09.577579 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8862d-e9ae-48ff-89e7-f84c7b984a08 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:09.579091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a8862d-e9ae-48ff-89e7-f84c7b984a08 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:09.582231 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a8862d-e9ae-48ff-89e7-f84c7b984a08 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:09.603745 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-b0a8862d-e9ae-48ff-89e7-f84c7b984a08 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 02:34:09.603745 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:09.603745 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 02:34:09.603745 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 02:34:09.603745 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 02:34:09.603745 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 02:34:09.603745 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 02:34:09.603745 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:34:09.603745 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 02:34:09.603745 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:34:09.603745 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 02:34:09.603745 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:34:09.603745 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:09.603745 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:09.603745 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:09.603745 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:09.603745 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:09.604940 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:09.604940 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:09.604940 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:09.604940 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:09.604940 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:09.604940 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:09.604940 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 02:34:09.604940 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 02:34:09.604940 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 02:34:09.604940 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 02:34:09.604940 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:09.604940 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:09.604940 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:09.604940 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:09.604940 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:09.604940 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:09.605898 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:09.605898 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:09.605898 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:09.605898 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:09.605898 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:09.605898 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:09.605898 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:09.605898 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:09.605898 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:09.605898 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:09.605898 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 02:34:09.605898 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:34:09.605898 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 02:34:09.605898 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:34:09.606724 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 468/951] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Sep 22 02:34:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:34:09.612325 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:09.615917 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180f56f-2a87-4a00-be36-31d8edb695f9 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:09.616436 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180f56f-2a87-4a00-be36-31d8edb695f9 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:09.617719 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180f56f-2a87-4a00-be36-31d8edb695f9 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:09.627573 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 469/952] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:34:09 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:34:09.629156 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180f56f-2a87-4a00-be36-31d8edb695f9 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:09.637808 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 484/953] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:34:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 02:34:09.639670 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78df71-33a3-4122-907a-3e3aacb14ee4 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:09.640059 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78df71-33a3-4122-907a-3e3aacb14ee4 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:09.640408 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78df71-33a3-4122-907a-3e3aacb14ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:09.668417 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-8a78df71-33a3-4122-907a-3e3aacb14ee4 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:34:09.670410 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-8a78df71-33a3-4122-907a-3e3aacb14ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:34:09.770995 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 485/954] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:34:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:34:09.779134 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180f56f-2a87-4a00-be36-31d8edb695f9 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:09.828288 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96101fe-2a21-4120-87ed-1c9baaf18a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:09.829079 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96101fe-2a21-4120-87ed-1c9baaf18a21 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:09.829388 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96101fe-2a21-4120-87ed-1c9baaf18a21 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:09.829673 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96101fe-2a21-4120-87ed-1c9baaf18a21 None admin] PATH_INFO: `/v3/services` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:09.830485 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96101fe-2a21-4120-87ed-1c9baaf18a21 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:09.831020 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96101fe-2a21-4120-87ed-1c9baaf18a21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:09.833189 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96101fe-2a21-4120-87ed-1c9baaf18a21 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:09.840217 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 470/955] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Sep 22 02:34:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:09.847456 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78df71-33a3-4122-907a-3e3aacb14ee4 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:09.955188 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78df71-33a3-4122-907a-3e3aacb14ee4 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:09.974236 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:09.994088 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51bd1c5-c62f-4888-8cc7-aaad7f6a6f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:09.994719 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bd1c5-c62f-4888-8cc7-aaad7f6a6f82 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:09.995030 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bd1c5-c62f-4888-8cc7-aaad7f6a6f82 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:09.995305 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bd1c5-c62f-4888-8cc7-aaad7f6a6f82 None admin] PATH_INFO: `/v3/services` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:09.995949 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51bd1c5-c62f-4888-8cc7-aaad7f6a6f82 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:09.996252 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51bd1c5-c62f-4888-8cc7-aaad7f6a6f82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:09.997640 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51bd1c5-c62f-4888-8cc7-aaad7f6a6f82 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:10.003638 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 486/956] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Sep 22 02:34:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:10.010491 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96101fe-2a21-4120-87ed-1c9baaf18a21 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:10.037267 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:10.059390 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51bd1c5-c62f-4888-8cc7-aaad7f6a6f82 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:10.140545 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96101fe-2a21-4120-87ed-1c9baaf18a21 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:10.177487 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065f819b-93cc-4cd5-a5b5-e8f20888703e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:10.178127 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f819b-93cc-4cd5-a5b5-e8f20888703e None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:10.178444 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f819b-93cc-4cd5-a5b5-e8f20888703e None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:10.178695 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f819b-93cc-4cd5-a5b5-e8f20888703e None admin] PATH_INFO: `/v3/endpoints` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:10.179413 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065f819b-93cc-4cd5-a5b5-e8f20888703e None admin] RBAC: Adding query filter params (interface=public, service_id=225890a6728a41148f5021adaf9359a4, region_id=RegionOne) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:10.179727 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065f819b-93cc-4cd5-a5b5-e8f20888703e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:10.181119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065f819b-93cc-4cd5-a5b5-e8f20888703e None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:10.186688 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 471/957] 10.222.0.20 () {56 vars in 1251 bytes} [Thu Sep 22 02:34:10 2022] GET /identity/v3/endpoints?service_id=225890a6728a41148f5021adaf9359a4&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:34:10.188433 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065f819b-93cc-4cd5-a5b5-e8f20888703e None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:10.223194 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51bd1c5-c62f-4888-8cc7-aaad7f6a6f82 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:10.261816 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9dfde-5366-4288-9605-6998dece40d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:10.262590 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9dfde-5366-4288-9605-6998dece40d1 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:10.262916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9dfde-5366-4288-9605-6998dece40d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:10.263194 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9dfde-5366-4288-9605-6998dece40d1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:10.276477 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d9dfde-5366-4288-9605-6998dece40d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:34:10.283026 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-46d9dfde-5366-4288-9605-6998dece40d1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 02:34:10.283026 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:10.283026 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:10.283026 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:10.283026 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:10.283026 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:10.283026 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:10.283026 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:10.283026 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:10.283026 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:10.283026 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:10.283026 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:10.283026 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:34:10.283026 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:34:10.283026 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:34:10.283026 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:34:10.284051 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:34:10.284051 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:34:10.284051 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:34:10.284051 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:34:10.284051 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:10.284051 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:10.284051 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:10.284051 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:10.284051 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:10.284051 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:10.284051 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:10.284051 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:10.284051 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:10.284051 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:10.284051 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:10.284051 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:10.285050 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:10.285050 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:10.285050 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:10.285050 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:10.285050 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:34:10.285050 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:34:10.285050 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:34:10.285050 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:34:10.285050 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:34:10.285050 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:34:10.285050 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 02:34:10.285050 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:10.285050 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 487/958] 10.222.0.20 () {56 vars in 1120 bytes} [Thu Sep 22 02:34:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:34:10.286980 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9dfde-5366-4288-9605-6998dece40d1 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:10.292195 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:10.324788 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065f819b-93cc-4cd5-a5b5-e8f20888703e None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:10.333790 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:10.368419 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a2ec6a-b7f7-4473-afce-799191615e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:10.369053 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a2ec6a-b7f7-4473-afce-799191615e91 None admin] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:10.369334 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a2ec6a-b7f7-4473-afce-799191615e91 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:10.369603 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a2ec6a-b7f7-4473-afce-799191615e91 None admin] PATH_INFO: `/v3/services` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:10.370268 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a2ec6a-b7f7-4473-afce-799191615e91 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:10.371712 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a2ec6a-b7f7-4473-afce-799191615e91 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:10.393294 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 472/959] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Sep 22 02:34:10 2022] POST /identity/v3/services => generated 242 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:34:10.395460 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a2ec6a-b7f7-4473-afce-799191615e91 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:10.421564 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9dfde-5366-4288-9605-6998dece40d1 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:10.497997 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0df0a4-2c1c-4cbc-ab5c-049da1d7e959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:10.499585 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0df0a4-2c1c-4cbc-ab5c-049da1d7e959 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:10.500313 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0df0a4-2c1c-4cbc-ab5c-049da1d7e959 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:10.501024 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0df0a4-2c1c-4cbc-ab5c-049da1d7e959 None admin] PATH_INFO: `/v3/services` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:10.502340 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0df0a4-2c1c-4cbc-ab5c-049da1d7e959 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:10.505685 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0df0a4-2c1c-4cbc-ab5c-049da1d7e959 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:10.518914 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 488/960] 10.222.0.20 () {56 vars in 1087 bytes} [Thu Sep 22 02:34:10 2022] GET /identity/v3/services => generated 1727 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 02:34:10.626611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a2ec6a-b7f7-4473-afce-799191615e91 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:10.681991 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3797e92e-d592-46cb-826a-54942570e56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:10.682678 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797e92e-d592-46cb-826a-54942570e56c None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:10.682966 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797e92e-d592-46cb-826a-54942570e56c None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:10.683247 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797e92e-d592-46cb-826a-54942570e56c None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:10.683955 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3797e92e-d592-46cb-826a-54942570e56c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:10.684264 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3797e92e-d592-46cb-826a-54942570e56c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:10.685709 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3797e92e-d592-46cb-826a-54942570e56c None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:10.688827 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:10.697669 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 473/961] 10.222.0.20 () {56 vars in 1115 bytes} [Thu Sep 22 02:34:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:10.709027 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0df0a4-2c1c-4cbc-ab5c-049da1d7e959 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:10.900190 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:10.905558 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0df0a4-2c1c-4cbc-ab5c-049da1d7e959 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:10.967120 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3e028-41b3-4a56-8bd7-1260207511e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:10.968169 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3e028-41b3-4a56-8bd7-1260207511e6 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:10.970864 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3e028-41b3-4a56-8bd7-1260207511e6 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:10.970864 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3e028-41b3-4a56-8bd7-1260207511e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:10.990080 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f3e028-41b3-4a56-8bd7-1260207511e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:10.992848 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f3e028-41b3-4a56-8bd7-1260207511e6 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:11.021727 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-f6f3e028-41b3-4a56-8bd7-1260207511e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:34:11.021727 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:11.021727 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:11.021727 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:11.021727 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:11.021727 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:11.021727 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:11.021727 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:11.021727 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:11.021727 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:11.021727 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:11.021727 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:11.021727 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:34:11.021727 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:34:11.021727 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:34:11.021727 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:34:11.023840 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:11.023840 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:11.023840 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:11.023840 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:11.023840 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:11.023840 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:11.023840 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:11.023840 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:11.023840 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:11.023840 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:11.023840 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:11.023840 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:11.023840 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:11.023840 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:11.023840 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:11.023840 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:11.023840 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:34:11.027245 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:34:11.027245 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:34:11.027245 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:11.027245 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 489/962] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:34:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:34:11.033710 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3797e92e-d592-46cb-826a-54942570e56c None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:11.190244 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3797e92e-d592-46cb-826a-54942570e56c None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:11.228088 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e86def4-8eb9-4f34-ae1a-82036a603da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:11.228709 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86def4-8eb9-4f34-ae1a-82036a603da5 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:11.228993 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86def4-8eb9-4f34-ae1a-82036a603da5 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:11.229272 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86def4-8eb9-4f34-ae1a-82036a603da5 None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:11.229931 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e86def4-8eb9-4f34-ae1a-82036a603da5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:11.230236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e86def4-8eb9-4f34-ae1a-82036a603da5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:11.231648 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e86def4-8eb9-4f34-ae1a-82036a603da5 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:11.244529 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 474/963] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:34:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:11.253111 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3e028-41b3-4a56-8bd7-1260207511e6 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:11.431459 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3e028-41b3-4a56-8bd7-1260207511e6 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:11.494430 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b864f8c-cce8-498d-b6d3-a64394a9d1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:11.495414 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b864f8c-cce8-498d-b6d3-a64394a9d1bf None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:11.495905 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b864f8c-cce8-498d-b6d3-a64394a9d1bf None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:11.496394 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b864f8c-cce8-498d-b6d3-a64394a9d1bf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:11.511229 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b864f8c-cce8-498d-b6d3-a64394a9d1bf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:11.513930 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b864f8c-cce8-498d-b6d3-a64394a9d1bf None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:11.529499 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-7b864f8c-cce8-498d-b6d3-a64394a9d1bf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 02:34:11.529499 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:11.529499 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:11.529499 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:11.529499 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:11.529499 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:11.529499 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:11.529499 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:11.529499 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:11.529499 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:11.529499 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:11.529499 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:11.529499 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:34:11.529499 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:34:11.529499 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:34:11.529499 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:34:11.532997 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:11.532997 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:11.532997 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:34:11.532997 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:34:11.532997 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:34:11.532997 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:34:11.532997 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:11.532997 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:11.532997 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:11.532997 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:11.532997 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:11.532997 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:11.532997 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:11.532997 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:11.532997 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:11.532997 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:11.532997 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:11.532997 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:11.535735 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:11.535735 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:11.535735 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:34:11.535735 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:34:11.535735 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:34:11.535735 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:34:11.535735 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:34:11.535735 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:34:11.535735 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 02:34:11.535735 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:11.535735 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 490/964] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Sep 22 02:34:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:34:11.539773 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e86def4-8eb9-4f34-ae1a-82036a603da5 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:11.649618 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:11.730156 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e86def4-8eb9-4f34-ae1a-82036a603da5 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:11.797577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9858987-7710-4b50-9989-870e9e443190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:11.798814 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9858987-7710-4b50-9989-870e9e443190 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:11.799490 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9858987-7710-4b50-9989-870e9e443190 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:11.800388 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9858987-7710-4b50-9989-870e9e443190 None admin] PATH_INFO: `/v3/users` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:11.802403 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9858987-7710-4b50-9989-870e9e443190 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:11.803150 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9858987-7710-4b50-9989-870e9e443190 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:11.805667 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9858987-7710-4b50-9989-870e9e443190 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:11.831529 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 475/965] 10.222.0.20 () {56 vars in 1149 bytes} [Thu Sep 22 02:34:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:11.839090 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b864f8c-cce8-498d-b6d3-a64394a9d1bf None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:11.866712 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:11.938794 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512aeb4-f4e9-40de-b0db-30667268b62b None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:11.939468 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512aeb4-f4e9-40de-b0db-30667268b62b None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:11.939948 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512aeb4-f4e9-40de-b0db-30667268b62b None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:11.941894 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 476/966] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:34:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:34:11.947528 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd783c24-448c-4fd3-9e06-45a6fbf63578 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:11.947817 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd783c24-448c-4fd3-9e06-45a6fbf63578 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:11.948095 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd783c24-448c-4fd3-9e06-45a6fbf63578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:11.967915 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-fd783c24-448c-4fd3-9e06-45a6fbf63578 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:34:11.969616 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-fd783c24-448c-4fd3-9e06-45a6fbf63578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:34:12.000658 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b864f8c-cce8-498d-b6d3-a64394a9d1bf None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:12.062049 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4227ee-a4c4-4863-9f60-bb25eb4dcb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:12.063458 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4227ee-a4c4-4863-9f60-bb25eb4dcb81 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:12.063737 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4227ee-a4c4-4863-9f60-bb25eb4dcb81 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:12.064298 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4227ee-a4c4-4863-9f60-bb25eb4dcb81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:12.079911 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 477/967] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:34:11 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 02:34:12.186600 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4227ee-a4c4-4863-9f60-bb25eb4dcb81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:12.188177 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4227ee-a4c4-4863-9f60-bb25eb4dcb81 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:12.206137 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:12.316333 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 491/968] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:34:11 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 02:34:12.322014 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd783c24-448c-4fd3-9e06-45a6fbf63578 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:12.356126 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4227ee-a4c4-4863-9f60-bb25eb4dcb81 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:12.400520 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:12.484245 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd783c24-448c-4fd3-9e06-45a6fbf63578 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:12.523182 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4227ee-a4c4-4863-9f60-bb25eb4dcb81 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:12.530353 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b12e9c-0e96-4990-8366-31a63d2a6ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:12.531048 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b12e9c-0e96-4990-8366-31a63d2a6ce0 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:12.531344 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b12e9c-0e96-4990-8366-31a63d2a6ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:12.531612 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b12e9c-0e96-4990-8366-31a63d2a6ce0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:12.544745 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b12e9c-0e96-4990-8366-31a63d2a6ce0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:12.545366 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:12.547045 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b12e9c-0e96-4990-8366-31a63d2a6ce0 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:12.568576 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-72b12e9c-0e96-4990-8366-31a63d2a6ce0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:34:12.568576 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:12.568576 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:12.568576 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:12.568576 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:12.568576 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:12.568576 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:12.568576 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:12.568576 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:12.568576 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:12.568576 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:12.568576 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:12.568576 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:34:12.568576 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:34:12.568576 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:34:12.568576 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:34:12.569775 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:12.569775 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:12.569775 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:12.569775 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:12.569775 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:12.569775 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:12.569775 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:12.569775 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:12.569775 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:12.569775 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:12.569775 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:12.569775 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:12.569775 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:12.569775 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:12.569775 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:12.569775 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:12.569775 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:34:12.570773 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:34:12.570773 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:34:12.570773 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:34:12.572658 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 478/969] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:34:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:34:12.582747 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b12e9c-0e96-4990-8366-31a63d2a6ce0 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:12.599913 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf614b-b005-41fa-bde9-d41cbf045363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:12.601603 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf614b-b005-41fa-bde9-d41cbf045363 None admin] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:12.602048 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf614b-b005-41fa-bde9-d41cbf045363 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:12.602470 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf614b-b005-41fa-bde9-d41cbf045363 None admin] PATH_INFO: `/v3/users` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:12.604318 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf614b-b005-41fa-bde9-d41cbf045363 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:12.607939 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf614b-b005-41fa-bde9-d41cbf045363 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:12.612230 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: WARNING keystone.server.flask.common [None req-87cf614b-b005-41fa-bde9-d41cbf045363 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 22 02:34:12.631021 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.sql.core [None req-87cf614b-b005-41fa-bde9-d41cbf045363 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 22 02:34:12.631021 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 22 02:34:12.631021 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [parameters: {'user_id': '11e70bad26694dceae023cf52bd686e6', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 22 02:34:12.631021 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88064) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 22 02:34:12.654595 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-87cf614b-b005-41fa-bde9-d41cbf045363 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 22 02:34:12.654595 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:12.654595 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 22 02:34:12.654595 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 22 02:34:12.654595 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 22 02:34:12.654595 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 22 02:34:12.654595 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 22 02:34:12.654595 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application result = self._query(query) Sep 22 02:34:12.654595 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 22 02:34:12.654595 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application conn.query(q) Sep 22 02:34:12.654595 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 22 02:34:12.654595 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 22 02:34:12.654595 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 22 02:34:12.654595 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application result.read() Sep 22 02:34:12.654595 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 22 02:34:12.655895 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 22 02:34:12.655895 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 22 02:34:12.655895 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 22 02:34:12.655895 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 22 02:34:12.655895 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 22 02:34:12.655895 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 22 02:34:12.655895 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 22 02:34:12.655895 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 22 02:34:12.655895 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:12.655895 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 22 02:34:12.655895 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:12.655895 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:12.655895 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 22 02:34:12.655895 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 22 02:34:12.655895 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 22 02:34:12.655895 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 22 02:34:12.655895 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 22 02:34:12.655895 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application next(self.gen) Sep 22 02:34:12.656862 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 22 02:34:12.656862 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application yield resource Sep 22 02:34:12.656862 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 22 02:34:12.656862 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application next(self.gen) Sep 22 02:34:12.656862 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 22 02:34:12.656862 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application self.session.rollback() Sep 22 02:34:12.656862 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 22 02:34:12.656862 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application self.force_reraise() Sep 22 02:34:12.656862 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 22 02:34:12.656862 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise self.value Sep 22 02:34:12.656862 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 22 02:34:12.656862 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 22 02:34:12.656862 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 22 02:34:12.656862 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application session.commit() Sep 22 02:34:12.656862 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 22 02:34:12.656862 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 22 02:34:12.656862 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 22 02:34:12.657805 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application self._prepare_impl() Sep 22 02:34:12.657805 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 22 02:34:12.657805 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application self.session.flush() Sep 22 02:34:12.657805 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 22 02:34:12.657805 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application self._flush(objects) Sep 22 02:34:12.657805 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 22 02:34:12.657805 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 22 02:34:12.657805 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 22 02:34:12.657805 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application compat.raise_( Sep 22 02:34:12.657805 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 22 02:34:12.657805 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception Sep 22 02:34:12.657805 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 22 02:34:12.657805 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application flush_context.execute() Sep 22 02:34:12.657805 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 22 02:34:12.657805 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rec.execute(self) Sep 22 02:34:12.657805 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 22 02:34:12.657805 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 22 02:34:12.658715 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 22 02:34:12.658715 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 22 02:34:12.658715 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 22 02:34:12.658715 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 22 02:34:12.658715 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 22 02:34:12.658715 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 22 02:34:12.658715 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 22 02:34:12.658715 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 22 02:34:12.658715 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 22 02:34:12.658715 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 22 02:34:12.658715 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 22 02:34:12.658715 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 22 02:34:12.658715 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 22 02:34:12.658715 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 22 02:34:12.658715 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 22 02:34:12.658715 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception Sep 22 02:34:12.660747 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 22 02:34:12.660747 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 22 02:34:12.660747 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 22 02:34:12.660747 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 22 02:34:12.660747 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 22 02:34:12.660747 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application result = self._query(query) Sep 22 02:34:12.660747 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 22 02:34:12.660747 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application conn.query(q) Sep 22 02:34:12.660747 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 22 02:34:12.660747 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 22 02:34:12.660747 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 22 02:34:12.660747 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application result.read() Sep 22 02:34:12.660747 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 22 02:34:12.660747 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 22 02:34:12.660747 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 22 02:34:12.660747 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 22 02:34:12.662803 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 22 02:34:12.662803 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 22 02:34:12.662803 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 22 02:34:12.662803 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 22 02:34:12.662803 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 22 02:34:12.662803 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 22 02:34:12.662803 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [parameters: {'user_id': '11e70bad26694dceae023cf52bd686e6', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 22 02:34:12.662803 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 22 02:34:12.662803 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:12.662803 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 02:34:12.662803 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:12.662803 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:12.662803 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:12.662803 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:12.662803 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:12.665138 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:12.665138 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:12.665138 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:12.665138 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:12.665138 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:12.665138 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:12.665138 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:12.665138 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 22 02:34:12.665138 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 22 02:34:12.665138 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:12.665138 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:12.665138 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:34:12.665138 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:34:12.665138 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:34:12.665138 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:34:12.665138 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 22 02:34:12.665138 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 22 02:34:12.667380 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 22 02:34:12.667380 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 22 02:34:12.667380 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 22 02:34:12.667380 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 22 02:34:12.667380 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 22 02:34:12.667380 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:12.667380 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 492/970] 10.222.0.20 () {60 vars in 1131 bytes} [Thu Sep 22 02:34:12 2022] POST /identity/v3/users => generated 160 bytes in 301 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:34:12.667380 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf614b-b005-41fa-bde9-d41cbf045363 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:12.749230 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:12.754493 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b12e9c-0e96-4990-8366-31a63d2a6ce0 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:12.794508 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2a2838-1bde-49cd-8ea2-7c2b942bb9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:12.795352 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a2838-1bde-49cd-8ea2-7c2b942bb9ef None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:12.795739 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a2838-1bde-49cd-8ea2-7c2b942bb9ef None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:12.796141 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a2838-1bde-49cd-8ea2-7c2b942bb9ef None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:12.796922 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2a2838-1bde-49cd-8ea2-7c2b942bb9ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:12.797338 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2a2838-1bde-49cd-8ea2-7c2b942bb9ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:12.798863 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2a2838-1bde-49cd-8ea2-7c2b942bb9ef None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:12.818739 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 479/971] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:34:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:34:12.824693 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2a2838-1bde-49cd-8ea2-7c2b942bb9ef None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:12.878102 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf614b-b005-41fa-bde9-d41cbf045363 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:12.963937 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b320c906-53e4-4720-a045-a2c9e00a5ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:12.964983 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2a2838-1bde-49cd-8ea2-7c2b942bb9ef None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:12.965559 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b320c906-53e4-4720-a045-a2c9e00a5ea2 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:12.966237 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b320c906-53e4-4720-a045-a2c9e00a5ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:12.967289 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b320c906-53e4-4720-a045-a2c9e00a5ea2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:12.983877 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b320c906-53e4-4720-a045-a2c9e00a5ea2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:12.988753 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b320c906-53e4-4720-a045-a2c9e00a5ea2 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:13.010567 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9365e86-858e-4191-af23-4d1071f125f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:13.015928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-b320c906-53e4-4720-a045-a2c9e00a5ea2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 02:34:13.015928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:13.015928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:13.015928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:13.015928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:13.015928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:13.015928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:13.015928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:13.015928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:13.015928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:13.015928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:13.015928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:13.015928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:34:13.015928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:34:13.015928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:34:13.015928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:34:13.017994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:13.017994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:13.017994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:34:13.017994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:34:13.017994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:34:13.017994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:34:13.017994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:13.017994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:13.017994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:13.017994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:13.017994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:13.017994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:13.017994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:13.017994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:13.017994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:13.017994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:13.017994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:13.017994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:13.020131 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:13.020131 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:13.020131 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:34:13.020131 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:34:13.020131 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:34:13.020131 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:34:13.020131 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:34:13.020131 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:34:13.020131 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 02:34:13.020131 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:13.021186 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9365e86-858e-4191-af23-4d1071f125f2 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:13.021186 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9365e86-858e-4191-af23-4d1071f125f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:13.021186 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9365e86-858e-4191-af23-4d1071f125f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:13.024668 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 493/972] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:34:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:34:13.032563 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b320c906-53e4-4720-a045-a2c9e00a5ea2 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:13.035810 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9365e86-858e-4191-af23-4d1071f125f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:13.037413 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9365e86-858e-4191-af23-4d1071f125f2 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:13.052912 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-f9365e86-858e-4191-af23-4d1071f125f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:34:13.052912 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:13.052912 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:13.052912 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:13.052912 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:13.052912 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:13.052912 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:13.052912 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:13.052912 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:13.052912 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:13.052912 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:13.052912 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:13.052912 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:34:13.052912 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:34:13.052912 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:34:13.063321 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:34:13.063321 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:13.063321 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:13.063321 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:13.063321 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:13.063321 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:13.063321 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:13.063321 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:13.063321 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:13.063321 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:13.063321 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:13.063321 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:13.063321 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:13.063321 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:13.063321 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:13.063321 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:13.063321 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:13.065518 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:34:13.065518 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:34:13.065518 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:34:13.065518 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:34:13.065518 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:34:13.065518 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:34:13.065518 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:34:13.065518 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:34:13.065518 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 480/973] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:34:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:34:13.065518 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9365e86-858e-4191-af23-4d1071f125f2 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:13.119395 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:13.130934 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:13.220088 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9365e86-858e-4191-af23-4d1071f125f2 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:13.256260 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8587f2-8c48-411f-9c43-8fd859b46888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:13.256873 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8587f2-8c48-411f-9c43-8fd859b46888 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:13.257154 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8587f2-8c48-411f-9c43-8fd859b46888 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:13.257429 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8587f2-8c48-411f-9c43-8fd859b46888 None admin] PATH_INFO: `/v3/projects` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:13.258125 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8587f2-8c48-411f-9c43-8fd859b46888 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:13.258432 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8587f2-8c48-411f-9c43-8fd859b46888 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:13.259911 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8587f2-8c48-411f-9c43-8fd859b46888 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:13.271775 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 481/974] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Sep 22 02:34:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:34:13.273367 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c8851-8066-419a-aaf1-28fd640d209f None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:13.273658 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c8851-8066-419a-aaf1-28fd640d209f None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:13.273933 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c8851-8066-419a-aaf1-28fd640d209f None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:13.275344 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 482/975] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:34:13.276152 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b320c906-53e4-4720-a045-a2c9e00a5ea2 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:13.276259 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bcd48a-e7e8-4d9d-bc0c-04a6f8dc4150 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:13.276599 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bcd48a-e7e8-4d9d-bc0c-04a6f8dc4150 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:13.276877 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bcd48a-e7e8-4d9d-bc0c-04a6f8dc4150 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:13.282469 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 483/976] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:34:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:34:13.283824 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b1a70-e926-4f5b-b690-ad44fd69a0c3 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:13.284113 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b1a70-e926-4f5b-b690-ad44fd69a0c3 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:13.284388 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b1a70-e926-4f5b-b690-ad44fd69a0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:13.303458 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-4f1b1a70-e926-4f5b-b690-ad44fd69a0c3 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:34:13.304659 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-4f1b1a70-e926-4f5b-b690-ad44fd69a0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:34:13.328445 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43abb440-dc47-46af-87aa-17b9eae42f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:13.329410 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43abb440-dc47-46af-87aa-17b9eae42f97 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:13.329803 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43abb440-dc47-46af-87aa-17b9eae42f97 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:13.330180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43abb440-dc47-46af-87aa-17b9eae42f97 None admin] PATH_INFO: `/v3/users` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:13.331063 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43abb440-dc47-46af-87aa-17b9eae42f97 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:13.331471 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43abb440-dc47-46af-87aa-17b9eae42f97 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:13.333093 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43abb440-dc47-46af-87aa-17b9eae42f97 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:13.388702 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 494/977] 10.222.0.20 () {56 vars in 1101 bytes} [Thu Sep 22 02:34:13 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:13.388702 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43abb440-dc47-46af-87aa-17b9eae42f97 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:13.388983 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 484/978] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:34:13 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:34:13.388983 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e150fa-82a4-49d7-852e-48a2667d43bc None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:13.388983 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e150fa-82a4-49d7-852e-48a2667d43bc None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:13.388983 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e150fa-82a4-49d7-852e-48a2667d43bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:13.408161 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-b8e150fa-82a4-49d7-852e-48a2667d43bc None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:34:13.410128 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-b8e150fa-82a4-49d7-852e-48a2667d43bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:34:13.451356 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:13.495826 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 485/979] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:34:13 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:34:13.564014 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43abb440-dc47-46af-87aa-17b9eae42f97 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:13.567407 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:13.609887 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad20732-df25-47b0-a3b4-669cc207d16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:13.610872 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad20732-df25-47b0-a3b4-669cc207d16f None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:13.611251 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad20732-df25-47b0-a3b4-669cc207d16f None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:13.611606 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad20732-df25-47b0-a3b4-669cc207d16f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:13.612455 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad20732-df25-47b0-a3b4-669cc207d16f None admin] RBAC: Adding query filter params (role.id=daf8ca6bfd534c00806f8cdfeb3bac5d, scope.project.id=2a792f8dc45145d687ddd58c94d44b5e, user.id=9821de2a961f4ddab2fca8e45b74e3db) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:13.612877 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad20732-df25-47b0-a3b4-669cc207d16f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:13.614487 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad20732-df25-47b0-a3b4-669cc207d16f None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:13.627358 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 495/980] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:34:13 2022] GET /identity/v3/role_assignments?user.id=9821de2a961f4ddab2fca8e45b74e3db&scope.project.id=2a792f8dc45145d687ddd58c94d44b5e&role.id=daf8ca6bfd534c00806f8cdfeb3bac5d => generated 608 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:13.737570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e150fa-82a4-49d7-852e-48a2667d43bc None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:13.840013 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad20732-df25-47b0-a3b4-669cc207d16f None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:13.855246 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e150fa-82a4-49d7-852e-48a2667d43bc None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:13.898584 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108e1dbe-a66f-42f9-b166-30a28bcf5b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:13.899268 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e1dbe-a66f-42f9-b166-30a28bcf5b28 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:13.899555 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e1dbe-a66f-42f9-b166-30a28bcf5b28 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:13.899833 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e1dbe-a66f-42f9-b166-30a28bcf5b28 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:13.900428 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108e1dbe-a66f-42f9-b166-30a28bcf5b28 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:13.901813 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108e1dbe-a66f-42f9-b166-30a28bcf5b28 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:13.911368 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-108e1dbe-a66f-42f9-b166-30a28bcf5b28 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 02:34:13.911368 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:13.911368 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 02:34:13.911368 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 02:34:13.911368 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 02:34:13.911368 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 02:34:13.911368 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 02:34:13.911368 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:34:13.911368 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 02:34:13.911368 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:34:13.911368 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 02:34:13.911368 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:34:13.911368 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:13.911368 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:13.911368 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:13.911368 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:13.911368 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:13.912521 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:13.912521 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:13.912521 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:13.912521 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:13.912521 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:13.912521 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:13.912521 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 02:34:13.912521 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 02:34:13.912521 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 02:34:13.912521 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 02:34:13.912521 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:13.912521 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:13.912521 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:13.912521 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:13.912521 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:13.912521 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:13.913361 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:13.913361 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:13.913361 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:13.913361 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:13.913361 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:13.913361 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:13.913361 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:13.913361 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:13.913361 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:13.913361 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:13.913361 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 02:34:13.913361 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:34:13.913361 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 02:34:13.913361 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:34:13.913361 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 486/981] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:34:13 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:34:13.918650 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108e1dbe-a66f-42f9-b166-30a28bcf5b28 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:13.921558 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:13.984764 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad20732-df25-47b0-a3b4-669cc207d16f None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:14.024728 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5356f-b0c2-4d81-bade-06447372eb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:14.025757 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5356f-b0c2-4d81-bade-06447372eb2c None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:14.025757 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5356f-b0c2-4d81-bade-06447372eb2c None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:14.025931 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5356f-b0c2-4d81-bade-06447372eb2c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:14.026415 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa5356f-b0c2-4d81-bade-06447372eb2c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:14.028250 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa5356f-b0c2-4d81-bade-06447372eb2c None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:14.038154 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-0fa5356f-b0c2-4d81-bade-06447372eb2c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 02:34:14.038154 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:14.038154 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 02:34:14.038154 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 02:34:14.038154 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 02:34:14.038154 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 02:34:14.038154 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 02:34:14.038154 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:34:14.038154 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 02:34:14.038154 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:14.038154 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 02:34:14.038154 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:14.038154 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:14.038154 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:14.038154 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:14.038154 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:14.038154 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:14.043451 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:14.043451 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:14.043451 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:14.043451 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:14.043451 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:14.043451 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:14.043451 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 02:34:14.043451 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 02:34:14.043451 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 02:34:14.043451 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 02:34:14.043451 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:14.043451 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:14.043451 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:14.043451 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:14.043451 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:14.043451 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:14.045238 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:14.045238 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:14.045238 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:14.045238 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:14.045238 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:14.045238 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:14.045238 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:14.045238 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:14.045238 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:14.045238 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:14.045238 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 02:34:14.045238 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:34:14.045238 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 02:34:14.045238 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:14.045238 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 496/982] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Sep 22 02:34:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:34:14.047818 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5356f-b0c2-4d81-bade-06447372eb2c None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:14.068174 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108e1dbe-a66f-42f9-b166-30a28bcf5b28 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:14.077257 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:14.121408 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6403dcff-a3de-4e41-beb9-1af76f671f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:14.122623 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403dcff-a3de-4e41-beb9-1af76f671f1c None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:14.123216 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403dcff-a3de-4e41-beb9-1af76f671f1c None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:14.123775 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403dcff-a3de-4e41-beb9-1af76f671f1c None admin] PATH_INFO: `/v3/services` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:14.124903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6403dcff-a3de-4e41-beb9-1af76f671f1c None admin] RBAC: Adding query filter params (name=compute) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:14.125539 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6403dcff-a3de-4e41-beb9-1af76f671f1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:14.128107 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6403dcff-a3de-4e41-beb9-1af76f671f1c None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:14.141082 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 487/983] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Sep 22 02:34:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:34:14.143178 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6403dcff-a3de-4e41-beb9-1af76f671f1c None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:14.203997 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5356f-b0c2-4d81-bade-06447372eb2c None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:14.257072 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:14.258432 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bee364-6b17-4036-b241-ffe594796941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:14.259244 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bee364-6b17-4036-b241-ffe594796941 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:14.259703 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bee364-6b17-4036-b241-ffe594796941 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:14.260156 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bee364-6b17-4036-b241-ffe594796941 None admin] PATH_INFO: `/v3/services` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:14.261323 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bee364-6b17-4036-b241-ffe594796941 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:14.261596 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bee364-6b17-4036-b241-ffe594796941 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:14.263930 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bee364-6b17-4036-b241-ffe594796941 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:14.273072 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 497/984] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Sep 22 02:34:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:14.280264 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bee364-6b17-4036-b241-ffe594796941 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:14.324129 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6403dcff-a3de-4e41-beb9-1af76f671f1c None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:14.373516 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72682c43-b9ea-491e-96e0-4a29af063268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:14.374478 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72682c43-b9ea-491e-96e0-4a29af063268 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:14.374975 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72682c43-b9ea-491e-96e0-4a29af063268 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:14.375459 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72682c43-b9ea-491e-96e0-4a29af063268 None admin] PATH_INFO: `/v3/services` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:14.376538 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72682c43-b9ea-491e-96e0-4a29af063268 None admin] RBAC: Adding query filter params (type=compute) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:14.377039 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72682c43-b9ea-491e-96e0-4a29af063268 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:14.380109 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72682c43-b9ea-491e-96e0-4a29af063268 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:14.392496 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 488/985] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Sep 22 02:34:14 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:14.393811 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72682c43-b9ea-491e-96e0-4a29af063268 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:14.409402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:14.432007 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bee364-6b17-4036-b241-ffe594796941 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:14.479536 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b86b350-bb58-4a8f-9274-3ddbb1017eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:14.480403 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86b350-bb58-4a8f-9274-3ddbb1017eb4 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:14.480863 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86b350-bb58-4a8f-9274-3ddbb1017eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:14.481590 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86b350-bb58-4a8f-9274-3ddbb1017eb4 None admin] PATH_INFO: `/v3/services` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:14.482807 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b86b350-bb58-4a8f-9274-3ddbb1017eb4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:14.483354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b86b350-bb58-4a8f-9274-3ddbb1017eb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:14.485337 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b86b350-bb58-4a8f-9274-3ddbb1017eb4 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:14.499291 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 498/986] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Sep 22 02:34:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:34:14.500539 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b86b350-bb58-4a8f-9274-3ddbb1017eb4 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:14.541375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72682c43-b9ea-491e-96e0-4a29af063268 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:14.593925 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:14.603524 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91048edf-6608-4a04-8352-1bf5095e910e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:14.604487 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91048edf-6608-4a04-8352-1bf5095e910e None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:14.604948 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91048edf-6608-4a04-8352-1bf5095e910e None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:14.605413 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91048edf-6608-4a04-8352-1bf5095e910e None admin] PATH_INFO: `/v3/endpoints` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:14.606285 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91048edf-6608-4a04-8352-1bf5095e910e None admin] RBAC: Adding query filter params (interface=public, service_id=51b951757c714155bb18a9873feb4c45, region_id=RegionOne) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:14.606735 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91048edf-6608-4a04-8352-1bf5095e910e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:14.608825 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91048edf-6608-4a04-8352-1bf5095e910e None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:14.621511 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 489/987] 10.222.0.20 () {56 vars in 1251 bytes} [Thu Sep 22 02:34:14 2022] GET /identity/v3/endpoints?service_id=51b951757c714155bb18a9873feb4c45&interface=public®ion_id=RegionOne => generated 194 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:14.623660 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91048edf-6608-4a04-8352-1bf5095e910e None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:14.661625 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b86b350-bb58-4a8f-9274-3ddbb1017eb4 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:14.711743 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fe3d9-130c-42dd-a3fc-90eacea4353b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:14.712380 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fe3d9-130c-42dd-a3fc-90eacea4353b None admin] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:14.712663 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fe3d9-130c-42dd-a3fc-90eacea4353b None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:14.712941 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fe3d9-130c-42dd-a3fc-90eacea4353b None admin] PATH_INFO: `/v3/endpoints` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:14.713623 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329fe3d9-130c-42dd-a3fc-90eacea4353b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:14.715094 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329fe3d9-130c-42dd-a3fc-90eacea4353b None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:14.747816 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 499/988] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Sep 22 02:34:14 2022] POST /identity/v3/endpoints => generated 349 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:34:14.780780 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91048edf-6608-4a04-8352-1bf5095e910e None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:14.843730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e440ec9-81f8-4537-a8fd-b547cb661da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:14.844960 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e440ec9-81f8-4537-a8fd-b547cb661da1 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:14.845508 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e440ec9-81f8-4537-a8fd-b547cb661da1 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:14.846005 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e440ec9-81f8-4537-a8fd-b547cb661da1 None admin] PATH_INFO: `/v3/services` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:14.846911 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e440ec9-81f8-4537-a8fd-b547cb661da1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:14.850217 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e440ec9-81f8-4537-a8fd-b547cb661da1 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:14.865622 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 490/989] 10.222.0.20 () {56 vars in 1087 bytes} [Thu Sep 22 02:34:14 2022] GET /identity/v3/services => generated 1727 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 02:34:15.928062 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d761d6-ae08-4b05-919b-5b5ba02b8d6c None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:15.928701 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d761d6-ae08-4b05-919b-5b5ba02b8d6c None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:15.928701 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d761d6-ae08-4b05-919b-5b5ba02b8d6c None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:15.930179 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 500/990] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:34:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:34:15.936508 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d6d53-83e0-4431-ac9a-32b913a8962b None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:15.937059 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d6d53-83e0-4431-ac9a-32b913a8962b None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:15.937543 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d6d53-83e0-4431-ac9a-32b913a8962b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:15.969844 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-ca7d6d53-83e0-4431-ac9a-32b913a8962b None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:34:15.971827 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-ca7d6d53-83e0-4431-ac9a-32b913a8962b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:34:16.064002 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:16.091487 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 491/991] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:34:15 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 02:34:16.364079 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d761d6-ae08-4b05-919b-5b5ba02b8d6c None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:16.368278 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:16.524071 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d761d6-ae08-4b05-919b-5b5ba02b8d6c None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:16.575769 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229150d-50b4-4d59-bf33-26561a32a69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:16.576431 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229150d-50b4-4d59-bf33-26561a32a69c None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:16.576741 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229150d-50b4-4d59-bf33-26561a32a69c None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:16.577018 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229150d-50b4-4d59-bf33-26561a32a69c None admin] PATH_INFO: `/v3/services/image` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:16.577617 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3229150d-50b4-4d59-bf33-26561a32a69c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:16.579093 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3229150d-50b4-4d59-bf33-26561a32a69c None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:16.589976 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-3229150d-50b4-4d59-bf33-26561a32a69c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 02:34:16.589976 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:16.589976 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 02:34:16.589976 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 02:34:16.589976 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 02:34:16.589976 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 02:34:16.589976 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 02:34:16.589976 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:34:16.589976 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 02:34:16.589976 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:16.589976 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 02:34:16.589976 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:16.589976 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:16.589976 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:16.589976 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:16.589976 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:16.589976 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:16.591843 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:16.591843 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:16.591843 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:16.591843 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:16.591843 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:16.591843 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:16.591843 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 02:34:16.591843 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 02:34:16.591843 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 02:34:16.591843 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 02:34:16.591843 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:16.591843 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:16.591843 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:16.591843 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:16.591843 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:16.591843 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:16.593452 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:16.593452 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:16.593452 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:16.593452 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:16.593452 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:16.593452 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:16.593452 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:16.593452 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:16.593452 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:16.593452 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:16.593452 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 02:34:16.593452 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:34:16.593452 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 02:34:16.593452 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:16.602017 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 501/992] 10.222.0.20 () {56 vars in 1105 bytes} [Thu Sep 22 02:34:16 2022] GET /identity/v3/services/image => generated 86 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:34:16.602409 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d6d53-83e0-4431-ac9a-32b913a8962b None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:16.739825 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d6d53-83e0-4431-ac9a-32b913a8962b None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:16.793814 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c11711-520e-412e-bfc3-881213e20655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:16.794882 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c11711-520e-412e-bfc3-881213e20655 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:16.795381 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c11711-520e-412e-bfc3-881213e20655 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:16.795866 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c11711-520e-412e-bfc3-881213e20655 None admin] PATH_INFO: `/v3/services` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:16.797018 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c11711-520e-412e-bfc3-881213e20655 None admin] RBAC: Adding query filter params (name=image) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:16.797564 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c11711-520e-412e-bfc3-881213e20655 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:16.800251 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c11711-520e-412e-bfc3-881213e20655 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:16.810897 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 492/993] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:34:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:34:16.818635 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229150d-50b4-4d59-bf33-26561a32a69c None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:16.930244 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:16.992366 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229150d-50b4-4d59-bf33-26561a32a69c None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:17.056285 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957e534-b1ce-4c2c-b4bf-e2f98696fc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:17.057342 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957e534-b1ce-4c2c-b4bf-e2f98696fc43 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:17.057833 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957e534-b1ce-4c2c-b4bf-e2f98696fc43 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:17.058333 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957e534-b1ce-4c2c-b4bf-e2f98696fc43 None admin] PATH_INFO: `/v3/services` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:17.059590 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b957e534-b1ce-4c2c-b4bf-e2f98696fc43 None admin] RBAC: Adding query filter params (type=image) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:17.060116 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b957e534-b1ce-4c2c-b4bf-e2f98696fc43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:17.062816 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b957e534-b1ce-4c2c-b4bf-e2f98696fc43 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:17.079172 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 502/994] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:34:16 2022] GET /identity/v3/services?type=image => generated 122 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:34:17.174177 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344f1f4-2f7f-46a3-9ed7-b057f2b37829 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:17.174729 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344f1f4-2f7f-46a3-9ed7-b057f2b37829 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:17.175313 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344f1f4-2f7f-46a3-9ed7-b057f2b37829 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:17.177913 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 493/995] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:34:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:34:17.184141 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0548da-aba1-4d27-8a19-9e2c2b56c18f None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:17.184406 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0548da-aba1-4d27-8a19-9e2c2b56c18f None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:17.184753 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0548da-aba1-4d27-8a19-9e2c2b56c18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:17.205276 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-1d0548da-aba1-4d27-8a19-9e2c2b56c18f None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:34:17.207500 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-1d0548da-aba1-4d27-8a19-9e2c2b56c18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:34:17.214508 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541012c-59db-4007-ae8e-68202ef405cb None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:17.214765 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541012c-59db-4007-ae8e-68202ef405cb None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:17.215107 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541012c-59db-4007-ae8e-68202ef405cb None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:17.216748 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 494/996] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:34:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:34:17.222640 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8fd861-357d-4f63-ae85-068cd856306b None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:17.223044 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8fd861-357d-4f63-ae85-068cd856306b None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:17.223198 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8fd861-357d-4f63-ae85-068cd856306b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:17.242987 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-4e8fd861-357d-4f63-ae85-068cd856306b None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:34:17.244119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-4e8fd861-357d-4f63-ae85-068cd856306b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:34:17.297308 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:17.303037 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 503/997] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:34:17 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 02:34:17.334962 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 495/998] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:34:17 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 02:34:17.578270 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0548da-aba1-4d27-8a19-9e2c2b56c18f None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:17.610632 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8fd861-357d-4f63-ae85-068cd856306b None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:17.673979 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:17.728928 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8fd861-357d-4f63-ae85-068cd856306b None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:17.729770 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0548da-aba1-4d27-8a19-9e2c2b56c18f None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:17.767101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da724b6-8da6-44d4-b212-197b9d85196a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:17.767742 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da724b6-8da6-44d4-b212-197b9d85196a None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:17.768023 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da724b6-8da6-44d4-b212-197b9d85196a None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:17.768297 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da724b6-8da6-44d4-b212-197b9d85196a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:17.768930 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da724b6-8da6-44d4-b212-197b9d85196a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:17.770308 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da724b6-8da6-44d4-b212-197b9d85196a None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:17.780070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-9da724b6-8da6-44d4-b212-197b9d85196a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 02:34:17.780070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:17.780070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 02:34:17.780070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 02:34:17.780070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 02:34:17.780070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 02:34:17.780070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 02:34:17.780070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:34:17.780070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 02:34:17.780070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:34:17.780070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 02:34:17.780070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:34:17.780070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:17.780070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:17.780070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:17.780070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:17.780070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:17.781222 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:17.781222 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:17.781222 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:17.781222 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:17.781222 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:17.781222 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:17.781222 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 02:34:17.781222 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 02:34:17.781222 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 02:34:17.781222 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 02:34:17.781222 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:17.781222 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:17.781222 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:17.781222 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:17.781222 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:17.781222 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:17.782075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:17.782075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:17.782075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:17.782075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:17.782075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:17.782075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:17.782075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:17.782075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:17.782075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:17.782075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:17.782075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 02:34:17.782075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:34:17.782075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 02:34:17.782075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:34:17.782075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 496/999] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Sep 22 02:34:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:34:17.792526 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316a6174-d331-4766-9903-1d466f6571ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:17.793170 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a6174-d331-4766-9903-1d466f6571ba None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:17.793453 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a6174-d331-4766-9903-1d466f6571ba None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:17.793728 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a6174-d331-4766-9903-1d466f6571ba None admin] PATH_INFO: `/v3/services/compute` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:17.794316 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316a6174-d331-4766-9903-1d466f6571ba None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:17.795719 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316a6174-d331-4766-9903-1d466f6571ba None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:17.800250 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da724b6-8da6-44d4-b212-197b9d85196a None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:17.805675 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-316a6174-d331-4766-9903-1d466f6571ba None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 02:34:17.805675 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:17.805675 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 02:34:17.805675 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 02:34:17.805675 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 02:34:17.805675 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 02:34:17.805675 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 02:34:17.805675 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:34:17.805675 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 02:34:17.805675 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:17.805675 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 02:34:17.805675 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:17.805675 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:17.805675 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:17.805675 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:17.805675 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:17.805675 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:17.806735 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:17.806735 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:17.806735 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:17.806735 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:17.806735 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:17.806735 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:17.806735 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 02:34:17.806735 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 02:34:17.806735 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 02:34:17.806735 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 02:34:17.806735 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:17.806735 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:17.806735 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:17.806735 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:17.806735 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:17.806735 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:17.808978 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:17.808978 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:17.808978 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:17.808978 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:17.808978 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:17.808978 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:17.808978 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:17.808978 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:17.808978 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:17.808978 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:17.808978 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 02:34:17.808978 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:34:17.808978 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 02:34:17.808978 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:17.808978 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 504/1000] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:34:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:34:17.819613 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316a6174-d331-4766-9903-1d466f6571ba None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:17.868699 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:17.947674 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316a6174-d331-4766-9903-1d466f6571ba None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:17.948770 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da724b6-8da6-44d4-b212-197b9d85196a None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:17.981627 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:17.992272 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3041b338-6379-4174-8908-a494390caa69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:17.993062 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041b338-6379-4174-8908-a494390caa69 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:17.993481 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041b338-6379-4174-8908-a494390caa69 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:17.993644 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f84421-961c-4f3d-a3e8-4ac9faff1f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:17.993958 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041b338-6379-4174-8908-a494390caa69 None admin] PATH_INFO: `/v3/services` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:17.994251 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f84421-961c-4f3d-a3e8-4ac9faff1f29 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:17.994538 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f84421-961c-4f3d-a3e8-4ac9faff1f29 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:17.994802 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f84421-961c-4f3d-a3e8-4ac9faff1f29 None admin] PATH_INFO: `/v3/services` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:17.995099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3041b338-6379-4174-8908-a494390caa69 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:17.995516 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3041b338-6379-4174-8908-a494390caa69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:17.995671 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f84421-961c-4f3d-a3e8-4ac9faff1f29 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:17.995810 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f84421-961c-4f3d-a3e8-4ac9faff1f29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:17.997175 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f84421-961c-4f3d-a3e8-4ac9faff1f29 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:17.997338 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3041b338-6379-4174-8908-a494390caa69 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:18.003654 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 505/1001] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Sep 22 02:34:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:34:18.008835 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 497/1002] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Sep 22 02:34:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:34:18.010277 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f84421-961c-4f3d-a3e8-4ac9faff1f29 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:18.012338 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3041b338-6379-4174-8908-a494390caa69 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:18.146327 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3041b338-6379-4174-8908-a494390caa69 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:18.147467 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f84421-961c-4f3d-a3e8-4ac9faff1f29 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:18.155169 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:18.195428 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348f064-1caf-4993-a742-a189adc293e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:18.196360 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348f064-1caf-4993-a742-a189adc293e5 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:18.196818 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348f064-1caf-4993-a742-a189adc293e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:18.197249 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348f064-1caf-4993-a742-a189adc293e5 None admin] PATH_INFO: `/v3/services` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:18.197357 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95aa4d-1d5c-486c-baf4-d297e979f6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:18.198229 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95aa4d-1d5c-486c-baf4-d297e979f6f3 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:18.198338 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8348f064-1caf-4993-a742-a189adc293e5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:18.198685 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95aa4d-1d5c-486c-baf4-d297e979f6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:18.198781 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8348f064-1caf-4993-a742-a189adc293e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:18.199152 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95aa4d-1d5c-486c-baf4-d297e979f6f3 None admin] PATH_INFO: `/v3/services` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:18.200157 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95aa4d-1d5c-486c-baf4-d297e979f6f3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:18.200656 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95aa4d-1d5c-486c-baf4-d297e979f6f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:18.201158 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8348f064-1caf-4993-a742-a189adc293e5 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:18.203029 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95aa4d-1d5c-486c-baf4-d297e979f6f3 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:18.215544 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 498/1003] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Sep 22 02:34:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:18.215807 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 506/1004] 10.222.0.20 () {56 vars in 1112 bytes} [Thu Sep 22 02:34:18 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:18.217139 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348f064-1caf-4993-a742-a189adc293e5 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:18.217328 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95aa4d-1d5c-486c-baf4-d297e979f6f3 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:18.295355 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:18.362737 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348f064-1caf-4993-a742-a189adc293e5 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:18.364938 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95aa4d-1d5c-486c-baf4-d297e979f6f3 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:18.403678 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9c31a-62e1-4315-9702-0906ee24a5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:18.404729 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9c31a-62e1-4315-9702-0906ee24a5b8 None admin] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:18.405243 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9c31a-62e1-4315-9702-0906ee24a5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:18.405742 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9c31a-62e1-4315-9702-0906ee24a5b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:18.406700 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc9c31a-62e1-4315-9702-0906ee24a5b8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:18.408280 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc9c31a-62e1-4315-9702-0906ee24a5b8 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:18.413054 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c51401-05f2-4649-8daf-10de7007f84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:18.414049 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c51401-05f2-4649-8daf-10de7007f84e None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:18.414537 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c51401-05f2-4649-8daf-10de7007f84e None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:18.415065 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c51401-05f2-4649-8daf-10de7007f84e None admin] PATH_INFO: `/v3/endpoints` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:18.416436 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c51401-05f2-4649-8daf-10de7007f84e None admin] RBAC: Adding query filter params (interface=admin, service_id=225890a6728a41148f5021adaf9359a4, region_id=RegionOne) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:18.417009 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c51401-05f2-4649-8daf-10de7007f84e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:18.419821 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c51401-05f2-4649-8daf-10de7007f84e None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:18.434414 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 507/1005] 10.222.0.20 () {56 vars in 1249 bytes} [Thu Sep 22 02:34:18 2022] GET /identity/v3/endpoints?service_id=225890a6728a41148f5021adaf9359a4&interface=admin®ion_id=RegionOne => generated 193 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:34:18.436194 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c51401-05f2-4649-8daf-10de7007f84e None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:18.446938 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 499/1006] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Sep 22 02:34:18 2022] POST /identity/v3/endpoints => generated 334 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 02:34:18.577945 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c51401-05f2-4649-8daf-10de7007f84e None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:18.600249 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:18.632287 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2ec50-9905-47c6-8c4d-c8812c28f8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:18.633190 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2ec50-9905-47c6-8c4d-c8812c28f8c9 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:18.633738 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2ec50-9905-47c6-8c4d-c8812c28f8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:18.634179 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2ec50-9905-47c6-8c4d-c8812c28f8c9 None admin] PATH_INFO: `/v3/services` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:18.635802 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e2ec50-9905-47c6-8c4d-c8812c28f8c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:18.637403 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e2ec50-9905-47c6-8c4d-c8812c28f8c9 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:18.646104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 508/1007] 10.222.0.20 () {56 vars in 1087 bytes} [Thu Sep 22 02:34:18 2022] GET /identity/v3/services => generated 1727 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 02:34:19.720236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaaea64-96d9-4f9f-8a78-ca413a991582 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:19.720914 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaaea64-96d9-4f9f-8a78-ca413a991582 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:19.721223 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaaea64-96d9-4f9f-8a78-ca413a991582 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:19.723772 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 500/1008] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:34:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:34:19.732479 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d297ac2-0945-4ac5-8445-c6c546e9f4a8 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:19.733102 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d297ac2-0945-4ac5-8445-c6c546e9f4a8 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:19.733411 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d297ac2-0945-4ac5-8445-c6c546e9f4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:19.764181 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-9d297ac2-0945-4ac5-8445-c6c546e9f4a8 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:34:19.766214 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-9d297ac2-0945-4ac5-8445-c6c546e9f4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:34:19.890917 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 509/1009] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:34:19 2022] POST /identity/v3/auth/tokens => generated 1726 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:34:20.390465 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaaea64-96d9-4f9f-8a78-ca413a991582 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:20.439713 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:20.534330 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaaea64-96d9-4f9f-8a78-ca413a991582 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:20.574227 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b858e-1f2f-4e43-a21b-e727cd41f5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:20.574891 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b858e-1f2f-4e43-a21b-e727cd41f5cd None admin] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:20.575171 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b858e-1f2f-4e43-a21b-e727cd41f5cd None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:20.575434 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b858e-1f2f-4e43-a21b-e727cd41f5cd None admin] PATH_INFO: `/v3/services` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:20.576135 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68b858e-1f2f-4e43-a21b-e727cd41f5cd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:20.577620 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68b858e-1f2f-4e43-a21b-e727cd41f5cd None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:20.598094 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 501/1010] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Sep 22 02:34:20 2022] POST /identity/v3/services => generated 242 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:34:20.887468 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce8059-0f6f-4e5a-8f14-cbfbaf18af4e None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:20.888049 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce8059-0f6f-4e5a-8f14-cbfbaf18af4e None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:20.888049 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce8059-0f6f-4e5a-8f14-cbfbaf18af4e None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:20.889534 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 510/1011] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:34:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:34:20.894923 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e86d8-03c5-4582-bccf-bc9c9b096a84 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:20.895580 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e86d8-03c5-4582-bccf-bc9c9b096a84 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:20.895580 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e86d8-03c5-4582-bccf-bc9c9b096a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:20.915592 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-5f9e86d8-03c5-4582-bccf-bc9c9b096a84 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:34:20.916814 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-5f9e86d8-03c5-4582-bccf-bc9c9b096a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:34:21.002904 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 502/1012] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:34:20 2022] POST /identity/v3/auth/tokens => generated 1822 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:34:21.259265 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892a20d-08c9-4f4e-877e-1d836e4cd833 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:21.259619 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892a20d-08c9-4f4e-877e-1d836e4cd833 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:21.259843 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892a20d-08c9-4f4e-877e-1d836e4cd833 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:21.261310 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 511/1013] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:34:21.267186 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329ceab-d1db-4c88-81ab-8e13b5ac3f65 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:21.267517 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329ceab-d1db-4c88-81ab-8e13b5ac3f65 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:21.267804 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329ceab-d1db-4c88-81ab-8e13b5ac3f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:21.270763 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0892a20d-08c9-4f4e-877e-1d836e4cd833 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:21.296330 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-4329ceab-d1db-4c88-81ab-8e13b5ac3f65 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:34:21.298358 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-4329ceab-d1db-4c88-81ab-8e13b5ac3f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:34:21.312226 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:21.407154 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0892a20d-08c9-4f4e-877e-1d836e4cd833 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:21.410962 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 503/1014] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:34:21 2022] POST /identity/v3/auth/tokens => generated 1822 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:34:21.424136 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:21.460237 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d405d3-9c3e-45de-a96b-f00f7ec2763e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:21.460880 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d405d3-9c3e-45de-a96b-f00f7ec2763e None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:21.461162 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d405d3-9c3e-45de-a96b-f00f7ec2763e None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:21.461440 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d405d3-9c3e-45de-a96b-f00f7ec2763e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:21.462054 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d405d3-9c3e-45de-a96b-f00f7ec2763e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:21.463491 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d405d3-9c3e-45de-a96b-f00f7ec2763e None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:21.475841 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-a4d405d3-9c3e-45de-a96b-f00f7ec2763e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 02:34:21.475841 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:21.475841 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 02:34:21.475841 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 02:34:21.475841 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 02:34:21.475841 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 02:34:21.475841 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 02:34:21.475841 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:34:21.475841 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 02:34:21.475841 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:21.475841 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 02:34:21.475841 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:21.475841 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:21.475841 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:21.475841 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:21.475841 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:21.475841 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:21.477559 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:21.477559 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:21.477559 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:21.477559 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:21.477559 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:21.477559 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:21.477559 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 02:34:21.477559 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 02:34:21.477559 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 02:34:21.477559 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 02:34:21.477559 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:21.477559 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:21.477559 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:21.477559 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:21.477559 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:21.477559 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:21.479685 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:21.479685 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:21.479685 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:21.479685 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:21.479685 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:21.479685 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:21.479685 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:21.479685 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:21.479685 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:21.479685 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:21.479685 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 02:34:21.479685 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:34:21.479685 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 02:34:21.479685 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:21.479685 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 512/1015] 10.222.0.20 () {56 vars in 1126 bytes} [Thu Sep 22 02:34:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:34:21.484318 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329ceab-d1db-4c88-81ab-8e13b5ac3f65 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:21.663206 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329ceab-d1db-4c88-81ab-8e13b5ac3f65 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:21.690790 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d405d3-9c3e-45de-a96b-f00f7ec2763e None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:21.724435 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e705a7-dfeb-4cab-baea-7dd97f7f3623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:21.725069 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e705a7-dfeb-4cab-baea-7dd97f7f3623 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:21.725363 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e705a7-dfeb-4cab-baea-7dd97f7f3623 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:21.725671 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e705a7-dfeb-4cab-baea-7dd97f7f3623 None admin] PATH_INFO: `/v3/services` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:21.726329 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e705a7-dfeb-4cab-baea-7dd97f7f3623 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:21.726650 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e705a7-dfeb-4cab-baea-7dd97f7f3623 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:21.728108 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e705a7-dfeb-4cab-baea-7dd97f7f3623 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:21.733586 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 504/1016] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Sep 22 02:34:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:21.740141 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e705a7-dfeb-4cab-baea-7dd97f7f3623 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:21.753421 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:21.856656 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d405d3-9c3e-45de-a96b-f00f7ec2763e None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:21.895410 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e705a7-dfeb-4cab-baea-7dd97f7f3623 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:21.912376 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f0f9b-6b50-4128-a371-5f0d9707a572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:21.913535 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f0f9b-6b50-4128-a371-5f0d9707a572 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:21.913913 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f0f9b-6b50-4128-a371-5f0d9707a572 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:21.914413 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f0f9b-6b50-4128-a371-5f0d9707a572 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:21.932277 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53f0f9b-6b50-4128-a371-5f0d9707a572 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:34:21.936236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93fed3-e685-4f91-b1ae-53bd9d79a246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:21.936953 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93fed3-e685-4f91-b1ae-53bd9d79a246 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:21.937238 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93fed3-e685-4f91-b1ae-53bd9d79a246 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:21.937516 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93fed3-e685-4f91-b1ae-53bd9d79a246 None admin] PATH_INFO: `/v3/services` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:21.938498 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d93fed3-e685-4f91-b1ae-53bd9d79a246 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:21.938910 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d93fed3-e685-4f91-b1ae-53bd9d79a246 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:21.940454 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d93fed3-e685-4f91-b1ae-53bd9d79a246 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:21.943412 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-b53f0f9b-6b50-4128-a371-5f0d9707a572 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 02:34:21.943412 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:21.943412 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:21.943412 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:21.943412 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:21.943412 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:21.943412 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:21.943412 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:21.943412 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:21.943412 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:21.943412 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:21.943412 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:21.943412 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:34:21.943412 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:34:21.943412 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:34:21.943412 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:34:21.944600 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:34:21.944600 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:34:21.944600 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:34:21.944600 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:34:21.944600 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:21.944600 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:21.944600 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:21.944600 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:21.944600 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:21.944600 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:21.944600 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:21.944600 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:21.944600 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:21.944600 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:21.944600 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:21.944600 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:21.945699 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:21.945699 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:21.945699 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:21.945699 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:21.945699 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:34:21.945699 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:34:21.945699 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:34:21.945699 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:34:21.945699 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:34:21.945699 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:34:21.945699 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 02:34:21.945699 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:21.945699 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 513/1017] 10.222.0.20 () {56 vars in 1120 bytes} [Thu Sep 22 02:34:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:34:21.951379 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f0f9b-6b50-4128-a371-5f0d9707a572 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:21.951566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 505/1018] 10.222.0.20 () {56 vars in 1122 bytes} [Thu Sep 22 02:34:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:34:21.957792 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93fed3-e685-4f91-b1ae-53bd9d79a246 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:21.998359 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:22.049897 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:22.088640 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93fed3-e685-4f91-b1ae-53bd9d79a246 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:22.100001 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f0f9b-6b50-4128-a371-5f0d9707a572 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:22.134321 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f309a-c17a-4641-90ac-48df9834c679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:22.135240 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f309a-c17a-4641-90ac-48df9834c679 None admin] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:22.135524 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f309a-c17a-4641-90ac-48df9834c679 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:22.135867 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f309a-c17a-4641-90ac-48df9834c679 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:22.136493 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f309a-c17a-4641-90ac-48df9834c679 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:22.137873 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f309a-c17a-4641-90ac-48df9834c679 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:22.139394 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7607aa0-7a55-4a55-9548-371df3a82b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:22.140084 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7607aa0-7a55-4a55-9548-371df3a82b2f None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:22.140302 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7607aa0-7a55-4a55-9548-371df3a82b2f None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:22.140583 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7607aa0-7a55-4a55-9548-371df3a82b2f None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:22.141260 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7607aa0-7a55-4a55-9548-371df3a82b2f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:22.141574 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7607aa0-7a55-4a55-9548-371df3a82b2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:22.142969 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7607aa0-7a55-4a55-9548-371df3a82b2f None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:22.153465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 514/1019] 10.222.0.20 () {56 vars in 1115 bytes} [Thu Sep 22 02:34:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:34:22.161905 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 506/1020] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Sep 22 02:34:21 2022] POST /identity/v3/endpoints => generated 325 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 02:34:22.167171 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7607aa0-7a55-4a55-9548-371df3a82b2f None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:22.358868 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:22.368108 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7607aa0-7a55-4a55-9548-371df3a82b2f None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:22.428821 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119e064-c390-4d0f-993c-a5fefb852879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:22.429410 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119e064-c390-4d0f-993c-a5fefb852879 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:22.429860 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119e064-c390-4d0f-993c-a5fefb852879 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:22.430147 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119e064-c390-4d0f-993c-a5fefb852879 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:22.446773 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119e064-c390-4d0f-993c-a5fefb852879 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:22.449092 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119e064-c390-4d0f-993c-a5fefb852879 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:22.471338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-f119e064-c390-4d0f-993c-a5fefb852879 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:34:22.471338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:22.471338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:22.471338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:22.471338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:22.471338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:22.471338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:22.471338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:22.471338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:22.471338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:22.471338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:22.471338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:22.471338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:34:22.471338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:34:22.471338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:34:22.471338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:34:22.473727 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:22.473727 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:22.473727 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:22.473727 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:22.473727 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:22.473727 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:22.473727 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:22.473727 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:22.473727 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:22.473727 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:22.473727 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:22.473727 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:22.473727 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:22.473727 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:22.473727 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:22.473727 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:22.473727 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:34:22.475426 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:34:22.475426 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:34:22.475426 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:22.475426 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 515/1021] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:34:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:34:22.481549 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f309a-c17a-4641-90ac-48df9834c679 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:22.647367 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f309a-c17a-4641-90ac-48df9834c679 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:22.693326 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:22.705248 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a0aa22-8202-4681-9316-9d89903b9c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:22.706937 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0aa22-8202-4681-9316-9d89903b9c8f None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:22.707428 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0aa22-8202-4681-9316-9d89903b9c8f None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:22.707851 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0aa22-8202-4681-9316-9d89903b9c8f None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:22.708787 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a0aa22-8202-4681-9316-9d89903b9c8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:22.709184 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a0aa22-8202-4681-9316-9d89903b9c8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:22.710888 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a0aa22-8202-4681-9316-9d89903b9c8f None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:22.731691 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 507/1022] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:34:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:22.738576 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119e064-c390-4d0f-993c-a5fefb852879 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:22.882569 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119e064-c390-4d0f-993c-a5fefb852879 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:22.922885 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabac56-16e3-4027-bbfa-6cf9fb236327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:22.923958 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabac56-16e3-4027-bbfa-6cf9fb236327 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:22.923958 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabac56-16e3-4027-bbfa-6cf9fb236327 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:22.924224 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabac56-16e3-4027-bbfa-6cf9fb236327 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:22.933222 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dabac56-16e3-4027-bbfa-6cf9fb236327 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:22.934706 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dabac56-16e3-4027-bbfa-6cf9fb236327 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:22.946173 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ce5a7-5099-45e1-8d2d-59d521df6834 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:22.946693 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ce5a7-5099-45e1-8d2d-59d521df6834 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:22.946693 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ce5a7-5099-45e1-8d2d-59d521df6834 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:22.948341 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 508/1023] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:34:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:34:22.948461 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-2dabac56-16e3-4027-bbfa-6cf9fb236327 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 02:34:22.948461 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:22.948461 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:22.948461 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:22.948461 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:22.948461 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:22.948461 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:22.948461 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:22.948461 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:22.948461 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:22.948461 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:22.948461 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:22.948461 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:34:22.948461 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:34:22.948461 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:34:22.948461 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:34:22.949380 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:22.949380 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:22.949380 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:34:22.949380 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:34:22.949380 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:34:22.949380 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:34:22.949380 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:22.949380 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:22.949380 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:22.949380 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:22.949380 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:22.949380 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:22.949380 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:22.949380 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:22.949380 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:22.949380 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:22.949380 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:22.949380 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:22.950364 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:22.950364 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:22.950364 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:34:22.950364 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:34:22.950364 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:34:22.950364 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:34:22.950364 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:34:22.950364 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:34:22.950364 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 02:34:22.950364 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:22.950364 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 516/1024] 10.222.0.20 () {56 vars in 1093 bytes} [Thu Sep 22 02:34:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:34:22.955009 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9074239-c92a-456c-8f65-7c92dc5bc92f None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:22.955389 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9074239-c92a-456c-8f65-7c92dc5bc92f None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:22.955669 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9074239-c92a-456c-8f65-7c92dc5bc92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:22.958105 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabac56-16e3-4027-bbfa-6cf9fb236327 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:22.962808 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:22.977931 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-d9074239-c92a-456c-8f65-7c92dc5bc92f None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:34:22.979424 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-d9074239-c92a-456c-8f65-7c92dc5bc92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:34:23.072124 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 509/1025] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:34:22 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:34:23.122425 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabac56-16e3-4027-bbfa-6cf9fb236327 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:23.166662 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4817e0e7-91de-42e9-a145-acdaf51948a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:23.167387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817e0e7-91de-42e9-a145-acdaf51948a8 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:23.167737 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817e0e7-91de-42e9-a145-acdaf51948a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:23.167996 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817e0e7-91de-42e9-a145-acdaf51948a8 None admin] PATH_INFO: `/v3/users` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:23.168721 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4817e0e7-91de-42e9-a145-acdaf51948a8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:23.169028 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4817e0e7-91de-42e9-a145-acdaf51948a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:23.170585 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4817e0e7-91de-42e9-a145-acdaf51948a8 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:23.193663 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 517/1026] 10.222.0.20 () {56 vars in 1133 bytes} [Thu Sep 22 02:34:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:34:23.199798 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9074239-c92a-456c-8f65-7c92dc5bc92f None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:23.319408 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9074239-c92a-456c-8f65-7c92dc5bc92f None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:23.343248 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:23.359268 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4817e0e7-91de-42e9-a145-acdaf51948a8 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:23.363876 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec4c59-5f8f-4ece-b082-6e0629ab087f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:23.364640 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec4c59-5f8f-4ece-b082-6e0629ab087f None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:23.365045 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec4c59-5f8f-4ece-b082-6e0629ab087f None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:23.365433 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec4c59-5f8f-4ece-b082-6e0629ab087f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:23.443636 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:23.450678 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ec4c59-5f8f-4ece-b082-6e0629ab087f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:23.452419 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ec4c59-5f8f-4ece-b082-6e0629ab087f None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:23.496811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4817e0e7-91de-42e9-a145-acdaf51948a8 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:23.547181 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d6bd5-5ad4-4e28-8d84-1143994e17d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:23.547955 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d6bd5-5ad4-4e28-8d84-1143994e17d9 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:23.548573 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d6bd5-5ad4-4e28-8d84-1143994e17d9 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:23.549046 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d6bd5-5ad4-4e28-8d84-1143994e17d9 None admin] PATH_INFO: `/v3/services/image` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:23.550351 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4d6bd5-5ad4-4e28-8d84-1143994e17d9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:23.552094 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4d6bd5-5ad4-4e28-8d84-1143994e17d9 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:23.563902 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-3e4d6bd5-5ad4-4e28-8d84-1143994e17d9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 02:34:23.563902 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:23.563902 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 02:34:23.563902 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 02:34:23.563902 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 02:34:23.563902 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 02:34:23.563902 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 02:34:23.563902 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:34:23.563902 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 02:34:23.563902 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:23.563902 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 02:34:23.563902 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:23.563902 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:23.563902 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:23.563902 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:23.563902 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:23.563902 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:23.566472 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:23.566472 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:23.566472 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:23.566472 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:23.566472 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:23.566472 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:23.566472 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 02:34:23.566472 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 02:34:23.566472 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 02:34:23.566472 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 02:34:23.566472 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:23.566472 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:23.566472 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:23.566472 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:23.566472 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:23.566472 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:23.568292 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:23.568292 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:23.568292 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:23.568292 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:23.568292 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:23.568292 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:23.568292 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:23.568292 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:23.568292 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:23.568292 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:23.568292 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 02:34:23.568292 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:34:23.568292 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 02:34:23.568292 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:23.568292 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 518/1027] 10.222.0.20 () {56 vars in 1105 bytes} [Thu Sep 22 02:34:23 2022] GET /identity/v3/services/image => generated 86 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:34:23.572393 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d6bd5-5ad4-4e28-8d84-1143994e17d9 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:23.585386 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 510/1028] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:34:23 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 02:34:23.588082 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec4c59-5f8f-4ece-b082-6e0629ab087f None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:23.613817 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:23.707387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d6bd5-5ad4-4e28-8d84-1143994e17d9 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:23.708862 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec4c59-5f8f-4ece-b082-6e0629ab087f None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:23.735893 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:23.748869 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df953d-90ce-4a15-806d-4df1525dd40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:23.749480 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df953d-90ce-4a15-806d-4df1525dd40b None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:23.749764 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df953d-90ce-4a15-806d-4df1525dd40b None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:23.750037 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df953d-90ce-4a15-806d-4df1525dd40b None admin] PATH_INFO: `/v3/services` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:23.750674 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1df953d-90ce-4a15-806d-4df1525dd40b None admin] RBAC: Adding query filter params (name=image) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:23.751012 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1df953d-90ce-4a15-806d-4df1525dd40b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:23.752493 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1df953d-90ce-4a15-806d-4df1525dd40b None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:23.763008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 519/1029] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:34:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:34:23.763187 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781fa315-46ec-4b4e-9c7c-cee8ddd1e8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:23.763187 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781fa315-46ec-4b4e-9c7c-cee8ddd1e8e9 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:23.763187 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781fa315-46ec-4b4e-9c7c-cee8ddd1e8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:23.763469 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781fa315-46ec-4b4e-9c7c-cee8ddd1e8e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:23.764072 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df953d-90ce-4a15-806d-4df1525dd40b None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:23.781268 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781fa315-46ec-4b4e-9c7c-cee8ddd1e8e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:23.784548 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781fa315-46ec-4b4e-9c7c-cee8ddd1e8e9 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:23.801917 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-781fa315-46ec-4b4e-9c7c-cee8ddd1e8e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:34:23.801917 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:23.801917 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:23.801917 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:23.801917 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:23.801917 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:23.801917 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:23.801917 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:23.801917 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:23.801917 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:23.801917 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:23.801917 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:23.801917 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:34:23.801917 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:34:23.801917 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:34:23.801917 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:34:23.802995 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:23.802995 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:23.802995 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:23.802995 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:23.802995 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:23.802995 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:23.802995 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:23.802995 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:23.802995 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:23.802995 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:23.802995 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:23.802995 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:23.802995 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:23.802995 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:23.802995 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:23.802995 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:23.802995 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:34:23.803916 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:34:23.803916 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:34:23.803916 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:34:23.805399 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 511/1030] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:34:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:34:23.814086 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781fa315-46ec-4b4e-9c7c-cee8ddd1e8e9 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:23.892387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df953d-90ce-4a15-806d-4df1525dd40b None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:23.920519 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:23.937007 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb405c5d-d17a-4e67-a342-74f5981bd6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:23.938185 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb405c5d-d17a-4e67-a342-74f5981bd6a8 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:23.938590 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb405c5d-d17a-4e67-a342-74f5981bd6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:23.939032 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb405c5d-d17a-4e67-a342-74f5981bd6a8 None admin] PATH_INFO: `/v3/services` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:23.939862 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb405c5d-d17a-4e67-a342-74f5981bd6a8 None admin] RBAC: Adding query filter params (type=image) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:23.940826 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb405c5d-d17a-4e67-a342-74f5981bd6a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:23.942450 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb405c5d-d17a-4e67-a342-74f5981bd6a8 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:23.953399 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 520/1031] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:34:23 2022] GET /identity/v3/services?type=image => generated 350 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:34:23.954735 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb405c5d-d17a-4e67-a342-74f5981bd6a8 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:23.970904 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781fa315-46ec-4b4e-9c7c-cee8ddd1e8e9 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:24.014515 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f0979-1065-4d30-8f21-2d52ba73f744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:24.015181 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f0979-1065-4d30-8f21-2d52ba73f744 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:24.015467 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f0979-1065-4d30-8f21-2d52ba73f744 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:24.015736 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f0979-1065-4d30-8f21-2d52ba73f744 None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:24.016461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634f0979-1065-4d30-8f21-2d52ba73f744 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:24.016771 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634f0979-1065-4d30-8f21-2d52ba73f744 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:24.018875 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634f0979-1065-4d30-8f21-2d52ba73f744 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:24.030466 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:24.039623 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 512/1032] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:34:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:34:24.045359 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f0979-1065-4d30-8f21-2d52ba73f744 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:24.092932 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb405c5d-d17a-4e67-a342-74f5981bd6a8 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:24.137573 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f3e3b-35de-4f61-b25e-c0781a89444f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:24.138658 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f3e3b-35de-4f61-b25e-c0781a89444f None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:24.138983 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f3e3b-35de-4f61-b25e-c0781a89444f None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:24.139264 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f3e3b-35de-4f61-b25e-c0781a89444f None admin] PATH_INFO: `/v3/endpoints` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:24.139980 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f3e3b-35de-4f61-b25e-c0781a89444f None admin] RBAC: Adding query filter params (interface=public, service_id=97dc2e5cc8884a27a88a3c3dee642179, region_id=RegionOne) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:24.140297 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f3e3b-35de-4f61-b25e-c0781a89444f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:24.142089 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f3e3b-35de-4f61-b25e-c0781a89444f None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:24.149474 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 521/1033] 10.222.0.20 () {56 vars in 1251 bytes} [Thu Sep 22 02:34:23 2022] GET /identity/v3/endpoints?service_id=97dc2e5cc8884a27a88a3c3dee642179&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:24.155403 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f3e3b-35de-4f61-b25e-c0781a89444f None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:24.173411 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f0979-1065-4d30-8f21-2d52ba73f744 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:24.217797 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:24.226078 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd191a0-d5b0-40fd-ad67-2d69978aac03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:24.226928 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd191a0-d5b0-40fd-ad67-2d69978aac03 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:24.227387 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd191a0-d5b0-40fd-ad67-2d69978aac03 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:24.227837 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd191a0-d5b0-40fd-ad67-2d69978aac03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:24.238812 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd191a0-d5b0-40fd-ad67-2d69978aac03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:24.241302 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd191a0-d5b0-40fd-ad67-2d69978aac03 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:24.255095 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-5dd191a0-d5b0-40fd-ad67-2d69978aac03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:34:24.255095 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:24.255095 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:24.255095 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:24.255095 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:24.255095 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:24.255095 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:24.255095 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:24.255095 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:24.255095 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:24.255095 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:24.255095 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:24.255095 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:34:24.255095 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:34:24.255095 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:34:24.256101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:34:24.256101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:24.256101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:24.256101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:24.256101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:24.256101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:24.256101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:24.256101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:24.256101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:24.256101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:24.256101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:24.256101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:24.256101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:24.256101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:24.256101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:24.256101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:24.256101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:24.257197 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:34:24.257197 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:34:24.257197 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:34:24.257197 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:34:24.257197 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:34:24.257197 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:34:24.257197 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:34:24.257197 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:34:24.264276 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 513/1034] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:34:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:34:24.265921 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd191a0-d5b0-40fd-ad67-2d69978aac03 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:24.304277 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f3e3b-35de-4f61-b25e-c0781a89444f None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:24.333491 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:24.370240 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e061618-e9b7-4a03-8fc2-8d94932615f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:24.371282 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e061618-e9b7-4a03-8fc2-8d94932615f9 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:24.371730 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e061618-e9b7-4a03-8fc2-8d94932615f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:24.372157 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e061618-e9b7-4a03-8fc2-8d94932615f9 None admin] PATH_INFO: `/v3/services` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:24.373083 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e061618-e9b7-4a03-8fc2-8d94932615f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:24.375350 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e061618-e9b7-4a03-8fc2-8d94932615f9 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:24.394867 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 522/1035] 10.222.0.20 () {56 vars in 1087 bytes} [Thu Sep 22 02:34:24 2022] GET /identity/v3/services => generated 1957 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 02:34:24.455317 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd191a0-d5b0-40fd-ad67-2d69978aac03 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:24.522873 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44759e63-0ee9-4226-bdc8-ee637ae8d265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:24.523836 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44759e63-0ee9-4226-bdc8-ee637ae8d265 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:24.524332 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44759e63-0ee9-4226-bdc8-ee637ae8d265 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:24.524797 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44759e63-0ee9-4226-bdc8-ee637ae8d265 None admin] PATH_INFO: `/v3/projects` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:24.526119 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44759e63-0ee9-4226-bdc8-ee637ae8d265 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:24.526655 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44759e63-0ee9-4226-bdc8-ee637ae8d265 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:24.529449 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44759e63-0ee9-4226-bdc8-ee637ae8d265 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:24.529649 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adce1e-bd9e-4840-bdfa-526c7e875641 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:24.529887 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adce1e-bd9e-4840-bdfa-526c7e875641 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:24.530317 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adce1e-bd9e-4840-bdfa-526c7e875641 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:24.532866 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 523/1036] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:34:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:34:24.542454 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036cfb1-587a-4298-af31-c153d6692fd4 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:24.542997 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036cfb1-587a-4298-af31-c153d6692fd4 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:24.543390 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036cfb1-587a-4298-af31-c153d6692fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:24.554236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 514/1037] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Sep 22 02:34:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:34:24.562641 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44759e63-0ee9-4226-bdc8-ee637ae8d265 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:24.567052 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-a036cfb1-587a-4298-af31-c153d6692fd4 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:34:24.568400 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-a036cfb1-587a-4298-af31-c153d6692fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:34:24.650614 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:24.659753 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 524/1038] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:34:24 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:34:24.733557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44759e63-0ee9-4226-bdc8-ee637ae8d265 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:24.792420 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1aa68c-6943-4dce-9585-914d98ff409d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:24.793373 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1aa68c-6943-4dce-9585-914d98ff409d None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:24.793869 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1aa68c-6943-4dce-9585-914d98ff409d None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:24.794351 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1aa68c-6943-4dce-9585-914d98ff409d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:24.795716 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1aa68c-6943-4dce-9585-914d98ff409d None admin] RBAC: Adding query filter params (role.id=daf8ca6bfd534c00806f8cdfeb3bac5d, scope.project.id=2a792f8dc45145d687ddd58c94d44b5e, user.id=a64ed6c8e15d479887ab1dc59997d715) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:24.796275 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1aa68c-6943-4dce-9585-914d98ff409d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:24.798983 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1aa68c-6943-4dce-9585-914d98ff409d None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:24.810903 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 515/1039] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:34:24 2022] GET /identity/v3/role_assignments?user.id=a64ed6c8e15d479887ab1dc59997d715&scope.project.id=2a792f8dc45145d687ddd58c94d44b5e&role.id=daf8ca6bfd534c00806f8cdfeb3bac5d => generated 259 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:34:24.927102 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036cfb1-587a-4298-af31-c153d6692fd4 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:25.045466 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036cfb1-587a-4298-af31-c153d6692fd4 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:25.060189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:25.092386 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2af7be-f1e9-4420-a435-1cec75ed0b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:25.093483 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2af7be-f1e9-4420-a435-1cec75ed0b83 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:25.093981 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2af7be-f1e9-4420-a435-1cec75ed0b83 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:25.094473 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2af7be-f1e9-4420-a435-1cec75ed0b83 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:25.096442 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2af7be-f1e9-4420-a435-1cec75ed0b83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:25.099196 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2af7be-f1e9-4420-a435-1cec75ed0b83 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:25.101940 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 525/1040] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:34:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:25.111421 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1aa68c-6943-4dce-9585-914d98ff409d None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:25.258553 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1aa68c-6943-4dce-9585-914d98ff409d None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:25.288242 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:25.312971 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de940c-a3eb-4a08-a8f5-77d855ae97d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:25.313567 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de940c-a3eb-4a08-a8f5-77d855ae97d5 None admin] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:25.313861 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de940c-a3eb-4a08-a8f5-77d855ae97d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:25.314135 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de940c-a3eb-4a08-a8f5-77d855ae97d5 None admin] PATH_INFO: `/v3/projects` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:25.314977 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34de940c-a3eb-4a08-a8f5-77d855ae97d5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:25.317527 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34de940c-a3eb-4a08-a8f5-77d855ae97d5 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:25.360214 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 516/1041] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Sep 22 02:34:25 2022] POST /identity/v3/projects => generated 311 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:34:26.973520 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff65d7e-1ceb-432e-9857-e0c68a17a7d4 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:26.974217 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff65d7e-1ceb-432e-9857-e0c68a17a7d4 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:26.974217 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff65d7e-1ceb-432e-9857-e0c68a17a7d4 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:26.975801 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 526/1042] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:34:26.982006 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95daf897-0377-4c68-a662-3781f1e922ab None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:26.982578 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95daf897-0377-4c68-a662-3781f1e922ab None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:26.982578 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95daf897-0377-4c68-a662-3781f1e922ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:27.004624 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-95daf897-0377-4c68-a662-3781f1e922ab None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:34:27.005847 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-95daf897-0377-4c68-a662-3781f1e922ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:34:27.093993 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65ee7a-d93c-4db5-88eb-9bc733a80c2a None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:27.094349 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65ee7a-d93c-4db5-88eb-9bc733a80c2a None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:27.094558 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65ee7a-d93c-4db5-88eb-9bc733a80c2a None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:27.095594 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 517/1043] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:34:26 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:34:27.096047 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 527/1044] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:34:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:34:27.101555 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe60e91-9d2e-4a16-9e51-3096149b024f None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:27.101946 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe60e91-9d2e-4a16-9e51-3096149b024f None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:27.102130 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe60e91-9d2e-4a16-9e51-3096149b024f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:27.122194 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-ebe60e91-9d2e-4a16-9e51-3096149b024f None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:34:27.123450 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-ebe60e91-9d2e-4a16-9e51-3096149b024f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:34:27.208989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 518/1045] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:34:27 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:34:27.369317 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a65ee7a-d93c-4db5-88eb-9bc733a80c2a None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:27.484370 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe60e91-9d2e-4a16-9e51-3096149b024f None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:27.507850 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a65ee7a-d93c-4db5-88eb-9bc733a80c2a None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:27.520039 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:27.554786 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407953b5-c2ad-4993-b8b4-3d85073ac210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:27.555493 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407953b5-c2ad-4993-b8b4-3d85073ac210 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:27.555807 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407953b5-c2ad-4993-b8b4-3d85073ac210 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:27.556067 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407953b5-c2ad-4993-b8b4-3d85073ac210 None admin] PATH_INFO: `/v3/services/image` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:27.556674 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407953b5-c2ad-4993-b8b4-3d85073ac210 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:27.558113 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407953b5-c2ad-4993-b8b4-3d85073ac210 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:27.568434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-407953b5-c2ad-4993-b8b4-3d85073ac210 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 02:34:27.568434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:27.568434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 02:34:27.568434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 02:34:27.568434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 02:34:27.568434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 02:34:27.568434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 02:34:27.568434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:34:27.568434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 02:34:27.568434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:27.568434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 02:34:27.568434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:27.568434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:27.568434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:27.568434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:27.568434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:27.568434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:27.570306 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:27.570306 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:27.570306 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:27.570306 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:27.570306 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:27.570306 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:27.570306 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 02:34:27.570306 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 02:34:27.570306 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 02:34:27.570306 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 02:34:27.570306 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:27.570306 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:27.570306 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:27.570306 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:27.570306 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:27.570306 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:27.571841 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:27.571841 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:27.571841 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:27.571841 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:27.571841 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:27.571841 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:27.571841 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:27.571841 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:27.571841 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:27.571841 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:27.571841 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 02:34:27.571841 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:34:27.571841 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 02:34:27.571841 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:27.571841 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 528/1046] 10.222.0.20 () {56 vars in 1105 bytes} [Thu Sep 22 02:34:27 2022] GET /identity/v3/services/image => generated 86 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:34:27.576101 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407953b5-c2ad-4993-b8b4-3d85073ac210 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:27.627220 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:27.640373 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe60e91-9d2e-4a16-9e51-3096149b024f None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:27.678193 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34087d98-9312-4ee7-be18-067b0b53c37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:27.678848 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34087d98-9312-4ee7-be18-067b0b53c37c None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:27.679163 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34087d98-9312-4ee7-be18-067b0b53c37c None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:27.679439 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34087d98-9312-4ee7-be18-067b0b53c37c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:27.691298 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34087d98-9312-4ee7-be18-067b0b53c37c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:34:27.696393 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-34087d98-9312-4ee7-be18-067b0b53c37c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 02:34:27.696393 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:27.696393 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:27.696393 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:27.696393 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:27.696393 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:27.696393 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:27.696393 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:27.696393 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:27.696393 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:27.696393 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:27.696393 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:27.696393 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:34:27.696393 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:34:27.696393 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:34:27.696393 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:34:27.697437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:34:27.697437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:34:27.697437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:34:27.697437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:34:27.697437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:27.697437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:27.697437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:27.697437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:27.697437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:27.697437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:27.697437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:27.697437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:27.697437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:27.697437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:27.697437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:27.697437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:27.698310 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:27.698310 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:27.698310 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:27.698310 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:27.698310 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:34:27.698310 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:34:27.698310 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:34:27.698310 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:34:27.698310 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:34:27.698310 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:34:27.698310 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 02:34:27.698310 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:34:27.698310 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 519/1047] 10.222.0.20 () {56 vars in 1120 bytes} [Thu Sep 22 02:34:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:34:27.699114 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a429a-c09a-43d6-a973-95e2d7ba78d2 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:27.699114 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a429a-c09a-43d6-a973-95e2d7ba78d2 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:27.699695 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a429a-c09a-43d6-a973-95e2d7ba78d2 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:27.701177 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 520/1048] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:34:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:34:27.707235 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcd1ce-f1ee-48f5-a731-57d48dc7542c None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:27.707529 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcd1ce-f1ee-48f5-a731-57d48dc7542c None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:27.707805 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcd1ce-f1ee-48f5-a731-57d48dc7542c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:27.728125 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-e2bcd1ce-f1ee-48f5-a731-57d48dc7542c None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:34:27.729343 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-e2bcd1ce-f1ee-48f5-a731-57d48dc7542c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:34:27.729965 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407953b5-c2ad-4993-b8b4-3d85073ac210 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:27.770102 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177fc46-4132-4d79-bf42-9f8a950bc50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:27.770730 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177fc46-4132-4d79-bf42-9f8a950bc50a None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:27.771041 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177fc46-4132-4d79-bf42-9f8a950bc50a None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:27.771315 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177fc46-4132-4d79-bf42-9f8a950bc50a None admin] PATH_INFO: `/v3/services` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:27.772270 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2177fc46-4132-4d79-bf42-9f8a950bc50a None admin] RBAC: Adding query filter params (name=image) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:27.772521 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2177fc46-4132-4d79-bf42-9f8a950bc50a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:27.774046 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2177fc46-4132-4d79-bf42-9f8a950bc50a None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:27.779977 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 529/1049] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:34:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:34:27.785998 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177fc46-4132-4d79-bf42-9f8a950bc50a None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:27.821925 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 521/1050] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:34:27 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:34:27.823029 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bcd1ce-f1ee-48f5-a731-57d48dc7542c None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:27.855416 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:27.953394 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177fc46-4132-4d79-bf42-9f8a950bc50a None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:27.967261 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:27.988218 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bcd1ce-f1ee-48f5-a731-57d48dc7542c None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:28.004255 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be554d04-6742-41e3-b086-6dfc32a679af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:28.005506 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be554d04-6742-41e3-b086-6dfc32a679af None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:28.005790 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be554d04-6742-41e3-b086-6dfc32a679af None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:28.006096 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be554d04-6742-41e3-b086-6dfc32a679af None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:28.006752 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be554d04-6742-41e3-b086-6dfc32a679af None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:28.007096 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be554d04-6742-41e3-b086-6dfc32a679af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:28.008478 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be554d04-6742-41e3-b086-6dfc32a679af None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:28.027123 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 530/1051] 10.222.0.20 () {56 vars in 1115 bytes} [Thu Sep 22 02:34:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:28.028438 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be554d04-6742-41e3-b086-6dfc32a679af None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:28.033531 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa0b03-dd52-4146-8117-093561dc29b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:28.034216 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa0b03-dd52-4146-8117-093561dc29b1 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:28.034499 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa0b03-dd52-4146-8117-093561dc29b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:28.034774 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa0b03-dd52-4146-8117-093561dc29b1 None admin] PATH_INFO: `/v3/services` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:28.035453 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aa0b03-dd52-4146-8117-093561dc29b1 None admin] RBAC: Adding query filter params (type=image) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:28.035784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aa0b03-dd52-4146-8117-093561dc29b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:28.037244 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aa0b03-dd52-4146-8117-093561dc29b1 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:28.047404 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 522/1052] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:34:27 2022] GET /identity/v3/services?type=image => generated 350 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:28.048305 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa0b03-dd52-4146-8117-093561dc29b1 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:28.151789 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:28.166589 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be554d04-6742-41e3-b086-6dfc32a679af None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:28.171911 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa0b03-dd52-4146-8117-093561dc29b1 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:28.203381 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d813e-7654-4012-8513-792a5ff9e75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:28.204238 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d813e-7654-4012-8513-792a5ff9e75a None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:28.204568 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d813e-7654-4012-8513-792a5ff9e75a None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:28.204945 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d813e-7654-4012-8513-792a5ff9e75a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:28.208654 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f439b99-48c0-451d-b2f7-20f7e532ec22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:28.209578 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f439b99-48c0-451d-b2f7-20f7e532ec22 None admin] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:28.210034 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f439b99-48c0-451d-b2f7-20f7e532ec22 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:28.210482 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f439b99-48c0-451d-b2f7-20f7e532ec22 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:28.211640 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f439b99-48c0-451d-b2f7-20f7e532ec22 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:28.214067 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f439b99-48c0-451d-b2f7-20f7e532ec22 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:28.223064 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545d813e-7654-4012-8513-792a5ff9e75a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:28.225595 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545d813e-7654-4012-8513-792a5ff9e75a None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:28.241622 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 523/1053] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Sep 22 02:34:28 2022] POST /identity/v3/endpoints => generated 327 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:34:28.242463 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f439b99-48c0-451d-b2f7-20f7e532ec22 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:28.242775 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-545d813e-7654-4012-8513-792a5ff9e75a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:34:28.242775 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:28.242775 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:28.242775 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:28.242775 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:28.242775 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:28.242775 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:28.242775 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:28.242775 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:28.242775 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:28.242775 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:28.242775 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:28.242775 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:34:28.242775 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:34:28.242775 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:34:28.242775 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:34:28.244562 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:28.244562 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:28.244562 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:28.244562 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:28.244562 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:28.244562 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:28.244562 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:28.244562 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:28.244562 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:28.244562 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:28.244562 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:28.244562 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:28.244562 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:28.244562 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:28.244562 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:28.244562 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:28.244562 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:34:28.246496 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:34:28.246496 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:34:28.246496 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:28.246496 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 531/1054] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:34:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:34:28.249833 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d813e-7654-4012-8513-792a5ff9e75a None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:28.289201 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:28.407782 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d813e-7654-4012-8513-792a5ff9e75a None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:28.418649 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f439b99-48c0-451d-b2f7-20f7e532ec22 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:28.460243 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90038f86-40b8-4937-9889-1575d3d772b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:28.461139 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90038f86-40b8-4937-9889-1575d3d772b7 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:28.461633 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90038f86-40b8-4937-9889-1575d3d772b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:28.461972 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90038f86-40b8-4937-9889-1575d3d772b7 None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:28.462707 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90038f86-40b8-4937-9889-1575d3d772b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:28.463218 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90038f86-40b8-4937-9889-1575d3d772b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:28.465152 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90038f86-40b8-4937-9889-1575d3d772b7 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:28.471915 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aebbfd-abd9-418a-b7f4-84057fd36ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:28.472582 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aebbfd-abd9-418a-b7f4-84057fd36ad4 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:28.472977 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aebbfd-abd9-418a-b7f4-84057fd36ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:28.473357 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aebbfd-abd9-418a-b7f4-84057fd36ad4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:28.474544 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aebbfd-abd9-418a-b7f4-84057fd36ad4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:28.476061 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aebbfd-abd9-418a-b7f4-84057fd36ad4 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:28.477842 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 524/1055] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:34:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:28.484050 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aebbfd-abd9-418a-b7f4-84057fd36ad4 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:28.488344 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:28.494378 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 532/1056] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:34:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:34:28.496299 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90038f86-40b8-4937-9889-1575d3d772b7 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:28.636999 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:28.648987 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aebbfd-abd9-418a-b7f4-84057fd36ad4 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:28.664186 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90038f86-40b8-4937-9889-1575d3d772b7 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:28.702206 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb6707-e168-4895-8da4-b1f04b041d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:28.703208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb6707-e168-4895-8da4-b1f04b041d16 None admin] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:28.703673 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb6707-e168-4895-8da4-b1f04b041d16 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:28.704120 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb6707-e168-4895-8da4-b1f04b041d16 None admin] PATH_INFO: `/v3/users` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:28.705273 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb6707-e168-4895-8da4-b1f04b041d16 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:28.708286 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb6707-e168-4895-8da4-b1f04b041d16 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:28.731604 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d93af1-6c72-4efa-9e29-9b34ada7d7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:28.732385 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d93af1-6c72-4efa-9e29-9b34ada7d7b2 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:28.732806 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d93af1-6c72-4efa-9e29-9b34ada7d7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:28.733257 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d93af1-6c72-4efa-9e29-9b34ada7d7b2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:28.744431 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d93af1-6c72-4efa-9e29-9b34ada7d7b2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:28.747698 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d93af1-6c72-4efa-9e29-9b34ada7d7b2 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:28.755301 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 525/1057] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Sep 22 02:34:28 2022] POST /identity/v3/users => generated 285 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:34:28.773386 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-98d93af1-6c72-4efa-9e29-9b34ada7d7b2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 02:34:28.773386 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:28.773386 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:28.773386 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:28.773386 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:28.773386 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:28.773386 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:28.773386 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:28.773386 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:28.773386 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:28.773386 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:28.773386 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:28.773386 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:34:28.773386 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:34:28.773386 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:34:28.773386 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:34:28.781777 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:28.781777 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:28.781777 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:34:28.781777 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:34:28.781777 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:34:28.781777 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:34:28.781777 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:28.781777 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:28.781777 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:28.781777 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:28.781777 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:28.781777 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:28.781777 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:28.781777 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:28.781777 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:28.781777 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:28.781777 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:28.781777 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:28.782948 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:28.782948 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:28.782948 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:34:28.782948 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:34:28.782948 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:34:28.782948 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:34:28.782948 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:34:28.782948 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:34:28.782948 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 02:34:28.782948 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:28.782948 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 533/1058] 10.222.0.20 () {56 vars in 1093 bytes} [Thu Sep 22 02:34:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:34:28.783574 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb6707-e168-4895-8da4-b1f04b041d16 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:28.899954 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:28.959251 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb6707-e168-4895-8da4-b1f04b041d16 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:29.003431 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d377a-7da3-477d-8c94-eefa92ba65af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:29.004493 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d377a-7da3-477d-8c94-eefa92ba65af None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:29.004962 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d377a-7da3-477d-8c94-eefa92ba65af None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:29.005463 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d377a-7da3-477d-8c94-eefa92ba65af None admin] PATH_INFO: `/v3/users` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:29.007028 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d377a-7da3-477d-8c94-eefa92ba65af None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:29.007588 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d377a-7da3-477d-8c94-eefa92ba65af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:29.010456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d377a-7da3-477d-8c94-eefa92ba65af None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:29.045963 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 526/1059] 10.222.0.20 () {56 vars in 1133 bytes} [Thu Sep 22 02:34:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:29.052810 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d93af1-6c72-4efa-9e29-9b34ada7d7b2 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:29.250922 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d93af1-6c72-4efa-9e29-9b34ada7d7b2 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:29.309794 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf52a19-0b27-441f-9c20-d2861f0d6487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:29.311209 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf52a19-0b27-441f-9c20-d2861f0d6487 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:29.311908 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf52a19-0b27-441f-9c20-d2861f0d6487 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:29.312609 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf52a19-0b27-441f-9c20-d2861f0d6487 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:29.329142 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:29.332930 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf52a19-0b27-441f-9c20-d2861f0d6487 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:29.334403 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf52a19-0b27-441f-9c20-d2861f0d6487 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:29.352425 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-faf52a19-0b27-441f-9c20-d2861f0d6487 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:34:29.352425 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:29.352425 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:29.352425 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:29.352425 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:29.352425 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:29.352425 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:29.352425 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:29.352425 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:29.352425 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:29.352425 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:29.352425 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:29.352425 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:34:29.352425 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:34:29.352425 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:34:29.352425 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:34:29.354210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:29.354210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:29.354210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:29.354210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:29.354210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:29.354210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:29.354210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:29.354210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:29.354210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:29.354210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:29.354210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:29.354210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:29.354210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:29.354210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:29.354210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:29.354210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:29.354210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:34:29.356006 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:34:29.356006 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:34:29.356006 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:29.356006 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 534/1060] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:34:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:34:29.359891 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d377a-7da3-477d-8c94-eefa92ba65af None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:29.544458 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d377a-7da3-477d-8c94-eefa92ba65af None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:29.570874 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:29.597221 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb30093-705a-4022-b39b-213f82b08e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:29.597914 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb30093-705a-4022-b39b-213f82b08e15 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:29.598375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb30093-705a-4022-b39b-213f82b08e15 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:29.598837 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb30093-705a-4022-b39b-213f82b08e15 None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:29.599755 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb30093-705a-4022-b39b-213f82b08e15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:29.600210 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb30093-705a-4022-b39b-213f82b08e15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:29.602438 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb30093-705a-4022-b39b-213f82b08e15 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:29.622926 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 527/1061] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:34:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:29.626218 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf52a19-0b27-441f-9c20-d2861f0d6487 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:29.800798 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf52a19-0b27-441f-9c20-d2861f0d6487 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:29.863079 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f6ed29-54c4-423e-8fda-588c20b9bb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:29.864123 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6ed29-54c4-423e-8fda-588c20b9bb48 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:29.864636 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6ed29-54c4-423e-8fda-588c20b9bb48 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:29.865137 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6ed29-54c4-423e-8fda-588c20b9bb48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:29.880196 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f6ed29-54c4-423e-8fda-588c20b9bb48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:29.883241 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f6ed29-54c4-423e-8fda-588c20b9bb48 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:29.908402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-e7f6ed29-54c4-423e-8fda-588c20b9bb48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:34:29.908402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:29.908402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:29.908402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:29.908402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:29.908402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:29.908402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:29.908402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:29.908402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:29.908402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:29.908402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:29.908402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:29.908402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:34:29.908402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:34:29.908402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:34:29.909454 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:34:29.909454 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:29.909454 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:29.909454 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:29.909454 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:29.909454 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:29.909454 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:29.909454 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:29.909454 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:29.909454 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:29.909454 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:29.909454 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:29.909454 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:29.909454 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:29.909454 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:29.909454 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:29.909454 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:29.910367 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:34:29.910367 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:34:29.910367 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:34:29.910367 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:34:29.910367 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:34:29.910367 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:34:29.910367 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:34:29.910367 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:29.910948 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 535/1062] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:34:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:34:29.921063 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb30093-705a-4022-b39b-213f82b08e15 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:30.091220 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb30093-705a-4022-b39b-213f82b08e15 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:30.131381 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09e28e-7b54-4084-bedc-9ffd684cb3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:30.131927 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09e28e-7b54-4084-bedc-9ffd684cb3df None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:30.132203 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09e28e-7b54-4084-bedc-9ffd684cb3df None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:30.132475 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09e28e-7b54-4084-bedc-9ffd684cb3df None admin] PATH_INFO: `/v3/projects` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:30.133264 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba09e28e-7b54-4084-bedc-9ffd684cb3df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:30.133578 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba09e28e-7b54-4084-bedc-9ffd684cb3df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:30.135049 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba09e28e-7b54-4084-bedc-9ffd684cb3df None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:30.149106 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 528/1063] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Sep 22 02:34:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:30.154962 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f6ed29-54c4-423e-8fda-588c20b9bb48 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:30.212245 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:30.265020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f6ed29-54c4-423e-8fda-588c20b9bb48 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:30.302325 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085479e-630b-4d7b-9178-a70aac123324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:30.303092 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085479e-630b-4d7b-9178-a70aac123324 None admin] REQUEST_METHOD: `PUT` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:30.303371 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085479e-630b-4d7b-9178-a70aac123324 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:30.303639 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085479e-630b-4d7b-9178-a70aac123324 None admin] PATH_INFO: `/v3/projects/2a792f8dc45145d687ddd58c94d44b5e/users/a64ed6c8e15d479887ab1dc59997d715/roles/daf8ca6bfd534c00806f8cdfeb3bac5d` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:30.305833 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8085479e-630b-4d7b-9178-a70aac123324 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a792f8dc45145d687ddd58c94d44b5e, user_id=a64ed6c8e15d479887ab1dc59997d715, role_id=daf8ca6bfd534c00806f8cdfeb3bac5d)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:30.307345 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8085479e-630b-4d7b-9178-a70aac123324 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:30.323688 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 536/1064] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Sep 22 02:34:30 2022] PUT /identity/v3/projects/2a792f8dc45145d687ddd58c94d44b5e/users/a64ed6c8e15d479887ab1dc59997d715/roles/daf8ca6bfd534c00806f8cdfeb3bac5d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 02:34:30.701183 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cfdf74-644f-4b36-aab6-3261c762736e None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:30.701586 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cfdf74-644f-4b36-aab6-3261c762736e None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:30.701786 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cfdf74-644f-4b36-aab6-3261c762736e None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:30.703394 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 529/1065] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:34:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:34:30.710746 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5bdf0-ced5-4a1f-be52-59e6e9951a0c None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:30.711124 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5bdf0-ced5-4a1f-be52-59e6e9951a0c None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:30.711357 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5bdf0-ced5-4a1f-be52-59e6e9951a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:30.738992 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-a7d5bdf0-ced5-4a1f-be52-59e6e9951a0c None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:34:30.740789 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-a7d5bdf0-ced5-4a1f-be52-59e6e9951a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:34:30.848155 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 537/1066] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:34:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:34:31.122781 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5fbb4-6d7a-4515-a8f3-95185f13219b None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:31.123618 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5fbb4-6d7a-4515-a8f3-95185f13219b None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:31.123999 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5fbb4-6d7a-4515-a8f3-95185f13219b None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:31.125492 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 530/1067] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:34:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:34:31.132481 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4392e1-0be6-441d-8f23-6096b7342aea None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:31.132946 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4392e1-0be6-441d-8f23-6096b7342aea None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:31.133381 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4392e1-0be6-441d-8f23-6096b7342aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:31.134583 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e5fbb4-6d7a-4515-a8f3-95185f13219b None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:31.155943 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-ff4392e1-0be6-441d-8f23-6096b7342aea None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:34:31.158248 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-ff4392e1-0be6-441d-8f23-6096b7342aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:34:31.187351 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:31.215250 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:31.279364 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 538/1068] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:34:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:34:31.309335 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e5fbb4-6d7a-4515-a8f3-95185f13219b None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:31.360591 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e477ff8-ba2a-4add-be09-40db16dc598a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:31.361247 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e477ff8-ba2a-4add-be09-40db16dc598a None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:31.361541 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e477ff8-ba2a-4add-be09-40db16dc598a None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:31.361841 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e477ff8-ba2a-4add-be09-40db16dc598a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:31.460002 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e477ff8-ba2a-4add-be09-40db16dc598a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:31.462271 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e477ff8-ba2a-4add-be09-40db16dc598a None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:31.508323 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:31.575084 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4392e1-0be6-441d-8f23-6096b7342aea None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:31.617582 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 531/1069] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:34:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 22 02:34:31.624060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e477ff8-ba2a-4add-be09-40db16dc598a None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:31.720216 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4392e1-0be6-441d-8f23-6096b7342aea None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:31.776194 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98231c-7086-4dc1-9ef7-066905b204ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:31.777215 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98231c-7086-4dc1-9ef7-066905b204ca None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:31.777683 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98231c-7086-4dc1-9ef7-066905b204ca None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:31.778144 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98231c-7086-4dc1-9ef7-066905b204ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:31.778882 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e477ff8-ba2a-4add-be09-40db16dc598a None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:31.797097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d98231c-7086-4dc1-9ef7-066905b204ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:34:31.817145 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-1d98231c-7086-4dc1-9ef7-066905b204ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:34:31.817145 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:31.817145 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:31.817145 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:31.817145 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:31.817145 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:31.817145 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:31.817145 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:31.817145 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:31.817145 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:31.817145 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:31.817145 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:31.817145 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:34:31.817145 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:34:31.817145 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:34:31.817145 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:34:31.818697 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:34:31.818697 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:34:31.818697 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:34:31.818697 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:34:31.818697 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:31.818697 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:31.818697 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:31.818697 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:31.818697 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:31.818697 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:31.818697 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:31.818697 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:31.818697 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:31.818697 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:31.818697 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:31.818697 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:31.819604 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:31.819604 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:31.819604 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:31.819604 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:31.819604 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:34:31.819604 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:34:31.819604 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:34:31.819604 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:34:31.819604 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:34:31.819604 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:34:31.819604 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:34:31.819604 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:31.819604 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 539/1070] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Sep 22 02:34:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:34:31.825515 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98231c-7086-4dc1-9ef7-066905b204ca None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:31.837363 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a972bc0-77d9-44f5-9fca-e0b33dfe9dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:31.838014 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a972bc0-77d9-44f5-9fca-e0b33dfe9dca None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:31.838298 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a972bc0-77d9-44f5-9fca-e0b33dfe9dca None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:31.838573 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a972bc0-77d9-44f5-9fca-e0b33dfe9dca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:31.841589 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:31.850141 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a972bc0-77d9-44f5-9fca-e0b33dfe9dca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:31.852143 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a972bc0-77d9-44f5-9fca-e0b33dfe9dca None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:31.863954 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:31.869900 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-9a972bc0-77d9-44f5-9fca-e0b33dfe9dca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:34:31.869900 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:31.869900 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:31.869900 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:31.869900 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:31.869900 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:31.869900 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:31.869900 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:31.869900 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:31.869900 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:31.869900 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:31.869900 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:31.869900 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:34:31.869900 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:34:31.869900 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:34:31.869900 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:34:31.871191 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:31.871191 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:31.871191 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:31.871191 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:31.871191 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:31.871191 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:31.871191 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:31.871191 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:31.871191 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:31.871191 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:31.871191 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:31.871191 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:31.871191 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:31.871191 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:31.871191 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:31.871191 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:31.871191 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:34:31.872128 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:34:31.872128 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:34:31.872128 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:34:31.874984 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 532/1071] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:34:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:34:31.878541 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a972bc0-77d9-44f5-9fca-e0b33dfe9dca None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:31.947911 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98231c-7086-4dc1-9ef7-066905b204ca None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:31.989372 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7c782-0bdc-471c-8077-659de4f75a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:31.989993 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7c782-0bdc-471c-8077-659de4f75a03 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:31.990272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7c782-0bdc-471c-8077-659de4f75a03 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:31.990543 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7c782-0bdc-471c-8077-659de4f75a03 None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:31.991256 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e7c782-0bdc-471c-8077-659de4f75a03 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:31.991560 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e7c782-0bdc-471c-8077-659de4f75a03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:31.992931 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e7c782-0bdc-471c-8077-659de4f75a03 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:31.993740 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a972bc0-77d9-44f5-9fca-e0b33dfe9dca None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:32.002037 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 540/1072] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:34:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:34:32.007944 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7c782-0bdc-471c-8077-659de4f75a03 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:32.038719 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafea11-8159-42bb-957e-90090e74d1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:32.039376 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafea11-8159-42bb-957e-90090e74d1d9 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:32.039662 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafea11-8159-42bb-957e-90090e74d1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:32.039944 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafea11-8159-42bb-957e-90090e74d1d9 None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:32.040598 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcafea11-8159-42bb-957e-90090e74d1d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:32.040905 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcafea11-8159-42bb-957e-90090e74d1d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:32.042314 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcafea11-8159-42bb-957e-90090e74d1d9 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:32.055682 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 533/1073] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:34:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:34:32.131873 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:32.143100 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7c782-0bdc-471c-8077-659de4f75a03 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:32.188946 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbf25c-1c5c-47de-b3b0-73cfc7c73549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:32.189646 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbf25c-1c5c-47de-b3b0-73cfc7c73549 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:32.189921 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbf25c-1c5c-47de-b3b0-73cfc7c73549 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:32.190198 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbf25c-1c5c-47de-b3b0-73cfc7c73549 None admin] PATH_INFO: `/v3/users/c02156f0a19a4996a77c3837d9876fd9` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:32.208332 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dbf25c-1c5c-47de-b3b0-73cfc7c73549 None admin] RBAC: Authorizing `identity:get_user(user_id=c02156f0a19a4996a77c3837d9876fd9)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:32.209798 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dbf25c-1c5c-47de-b3b0-73cfc7c73549 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:32.211811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 541/1074] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:34:32 2022] GET /identity/v3/users/c02156f0a19a4996a77c3837d9876fd9 => generated 285 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:32.218055 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafea11-8159-42bb-957e-90090e74d1d9 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:32.278592 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:32.341692 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafea11-8159-42bb-957e-90090e74d1d9 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:32.382227 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7fbbe-59eb-48a4-9ec4-e5d3639d579f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:32.382795 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7fbbe-59eb-48a4-9ec4-e5d3639d579f None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:32.383200 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7fbbe-59eb-48a4-9ec4-e5d3639d579f None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:32.383481 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7fbbe-59eb-48a4-9ec4-e5d3639d579f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:32.460811 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f7fbbe-59eb-48a4-9ec4-e5d3639d579f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:32.462285 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f7fbbe-59eb-48a4-9ec4-e5d3639d579f None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:32.515821 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:32.576285 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 534/1075] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:34:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 22 02:34:32.582603 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbf25c-1c5c-47de-b3b0-73cfc7c73549 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:32.709020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbf25c-1c5c-47de-b3b0-73cfc7c73549 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:32.752973 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed5df8-9946-42d8-9f5e-29ae50bf04aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:32.753621 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed5df8-9946-42d8-9f5e-29ae50bf04aa None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:32.753906 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed5df8-9946-42d8-9f5e-29ae50bf04aa None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:32.754188 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed5df8-9946-42d8-9f5e-29ae50bf04aa None admin] PATH_INFO: `/v3/projects/fd36fd0411884b52b3319ef42551a7ef` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:32.755374 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ed5df8-9946-42d8-9f5e-29ae50bf04aa None admin] RBAC: Authorizing `identity:get_project(project_id=fd36fd0411884b52b3319ef42551a7ef)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:32.756831 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ed5df8-9946-42d8-9f5e-29ae50bf04aa None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:32.758534 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 542/1076] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:34:32 2022] GET /identity/v3/projects/fd36fd0411884b52b3319ef42551a7ef => generated 311 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:34:32.760790 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452295c5-25f5-4f8f-b0f0-ac4f1bb7f8de None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:32.761083 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452295c5-25f5-4f8f-b0f0-ac4f1bb7f8de None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:32.761362 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452295c5-25f5-4f8f-b0f0-ac4f1bb7f8de None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:32.762719 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 535/1077] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:34:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:34:32.765183 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed5df8-9946-42d8-9f5e-29ae50bf04aa None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:32.768422 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff2a8a-a332-453f-a98d-9a7f75087478 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:32.768708 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff2a8a-a332-453f-a98d-9a7f75087478 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:32.768976 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff2a8a-a332-453f-a98d-9a7f75087478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:32.788242 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-41ff2a8a-a332-453f-a98d-9a7f75087478 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:34:32.789486 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-41ff2a8a-a332-453f-a98d-9a7f75087478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:34:32.795875 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:32.883720 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 536/1078] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:34:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 02:34:32.902124 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed5df8-9946-42d8-9f5e-29ae50bf04aa None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:32.940793 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d0f15-1b72-4991-b781-3c90bf1ae48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:32.941407 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d0f15-1b72-4991-b781-3c90bf1ae48c None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:32.941692 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d0f15-1b72-4991-b781-3c90bf1ae48c None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:32.941962 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d0f15-1b72-4991-b781-3c90bf1ae48c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:32.942692 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d0f15-1b72-4991-b781-3c90bf1ae48c None admin] RBAC: Adding query filter params (role.id=31a687548f55493b8ea666fe7e350303, scope.project.id=fd36fd0411884b52b3319ef42551a7ef, user.id=c02156f0a19a4996a77c3837d9876fd9) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:32.943030 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d0f15-1b72-4991-b781-3c90bf1ae48c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:32.944527 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d0f15-1b72-4991-b781-3c90bf1ae48c None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:32.950438 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 543/1079] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:34:32 2022] GET /identity/v3/role_assignments?user.id=c02156f0a19a4996a77c3837d9876fd9&scope.project.id=fd36fd0411884b52b3319ef42551a7ef&role.id=31a687548f55493b8ea666fe7e350303 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:33.151244 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff2a8a-a332-453f-a98d-9a7f75087478 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:33.239141 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:33.267635 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff2a8a-a332-453f-a98d-9a7f75087478 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:33.307554 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033ec8b9-3ccf-4c09-8856-68d2b06d8deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:33.308121 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ec8b9-3ccf-4c09-8856-68d2b06d8deb None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:33.308406 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ec8b9-3ccf-4c09-8856-68d2b06d8deb None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:33.308683 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ec8b9-3ccf-4c09-8856-68d2b06d8deb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:33.320706 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033ec8b9-3ccf-4c09-8856-68d2b06d8deb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:34:33.325918 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-033ec8b9-3ccf-4c09-8856-68d2b06d8deb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 02:34:33.325918 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:33.325918 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:33.325918 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:33.325918 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:33.325918 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:33.325918 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:33.325918 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:33.325918 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:33.325918 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:33.325918 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:33.325918 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:33.325918 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:34:33.325918 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:34:33.325918 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:34:33.325918 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:34:33.331070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:34:33.331070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:34:33.331070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:34:33.331070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:34:33.331070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:33.331070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:33.331070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:33.331070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:33.331070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:33.331070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:33.331070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:33.331070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:33.331070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:33.331070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:33.331070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:33.331070 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:33.331993 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:33.331993 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:33.331993 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:33.331993 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:33.331993 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:34:33.331993 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:34:33.331993 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:34:33.331993 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:34:33.331993 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:34:33.331993 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:34:33.331993 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 02:34:33.331993 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:34:33.331993 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 537/1080] 10.222.0.20 () {56 vars in 1120 bytes} [Thu Sep 22 02:34:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:34:33.334073 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d0f15-1b72-4991-b781-3c90bf1ae48c None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:33.438389 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:33.468452 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d0f15-1b72-4991-b781-3c90bf1ae48c None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:33.516728 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bbe1aa-a4e8-46c1-a18c-f0540c306215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:33.517351 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bbe1aa-a4e8-46c1-a18c-f0540c306215 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:33.517634 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bbe1aa-a4e8-46c1-a18c-f0540c306215 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:33.518523 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bbe1aa-a4e8-46c1-a18c-f0540c306215 None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:33.519182 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bbe1aa-a4e8-46c1-a18c-f0540c306215 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:33.519490 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bbe1aa-a4e8-46c1-a18c-f0540c306215 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:33.520867 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bbe1aa-a4e8-46c1-a18c-f0540c306215 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:33.530463 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 544/1081] 10.222.0.20 () {56 vars in 1115 bytes} [Thu Sep 22 02:34:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:33.537065 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033ec8b9-3ccf-4c09-8856-68d2b06d8deb None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:33.650027 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033ec8b9-3ccf-4c09-8856-68d2b06d8deb None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:33.690524 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d8c1e-b757-43a5-bf5a-1be2b5c87f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:33.691211 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d8c1e-b757-43a5-bf5a-1be2b5c87f6f None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:33.691496 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d8c1e-b757-43a5-bf5a-1be2b5c87f6f None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:33.691773 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d8c1e-b757-43a5-bf5a-1be2b5c87f6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:33.699392 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:33.703976 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8d8c1e-b757-43a5-bf5a-1be2b5c87f6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:33.706295 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8d8c1e-b757-43a5-bf5a-1be2b5c87f6f None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:33.723099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-3a8d8c1e-b757-43a5-bf5a-1be2b5c87f6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:34:33.723099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:33.723099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:33.723099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:33.723099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:33.723099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:33.723099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:33.723099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:33.723099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:33.723099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:33.723099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:33.723099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:33.723099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:34:33.723099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:34:33.723099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:34:33.723099 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:34:33.724503 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:33.724503 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:33.724503 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:33.724503 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:33.724503 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:33.724503 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:33.724503 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:33.724503 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:33.724503 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:33.724503 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:33.724503 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:33.724503 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:33.724503 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:33.724503 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:33.724503 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:33.724503 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:33.724503 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:34:33.726172 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:34:33.726172 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:34:33.726172 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:34:33.726172 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 538/1082] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:34:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:34:33.730088 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bbe1aa-a4e8-46c1-a18c-f0540c306215 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:33.847025 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bbe1aa-a4e8-46c1-a18c-f0540c306215 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:33.887140 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cccdac-8a04-4102-a43a-92766c7765c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:33.887762 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cccdac-8a04-4102-a43a-92766c7765c7 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:33.888040 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cccdac-8a04-4102-a43a-92766c7765c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:33.888314 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cccdac-8a04-4102-a43a-92766c7765c7 None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:33.888966 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cccdac-8a04-4102-a43a-92766c7765c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:33.889265 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cccdac-8a04-4102-a43a-92766c7765c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:33.890650 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cccdac-8a04-4102-a43a-92766c7765c7 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:33.904160 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 545/1083] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:34:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:33.909790 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d8c1e-b757-43a5-bf5a-1be2b5c87f6f None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:34.019287 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d8c1e-b757-43a5-bf5a-1be2b5c87f6f None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:34.056957 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3b2ee1-4d8b-497f-bbc4-bffb50f446f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:34.057582 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b2ee1-4d8b-497f-bbc4-bffb50f446f2 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:34.057868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b2ee1-4d8b-497f-bbc4-bffb50f446f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:34.058152 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b2ee1-4d8b-497f-bbc4-bffb50f446f2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:34.068101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3b2ee1-4d8b-497f-bbc4-bffb50f446f2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:34.069567 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3b2ee1-4d8b-497f-bbc4-bffb50f446f2 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:34.081981 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-2a3b2ee1-4d8b-497f-bbc4-bffb50f446f2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 02:34:34.081981 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:34.081981 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:34.081981 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:34.081981 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:34.081981 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:34.081981 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:34.081981 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:34.081981 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:34.081981 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:34.081981 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:34.081981 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:34.081981 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:34:34.081981 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:34:34.081981 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:34:34.081981 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:34:34.083060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:34.083060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:34.083060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:34:34.083060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:34:34.083060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:34:34.083060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:34:34.083060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:34.083060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:34.083060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:34.083060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:34.083060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:34.083060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:34.083060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:34.083060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:34.083060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:34.083060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:34.083060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:34.083060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:34.084018 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:34.084018 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:34.084018 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:34:34.084018 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:34:34.084018 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:34:34.084018 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:34:34.084018 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:34:34.084018 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:34:34.084018 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 02:34:34.084018 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:34:34.084018 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 539/1084] 10.222.0.20 () {56 vars in 1093 bytes} [Thu Sep 22 02:34:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:34:34.089660 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cccdac-8a04-4102-a43a-92766c7765c7 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:34.099424 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:34.208461 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cccdac-8a04-4102-a43a-92766c7765c7 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:34.246489 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936599a8-c08c-440d-bc81-4344da4e5128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:34.247135 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936599a8-c08c-440d-bc81-4344da4e5128 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:34.247422 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936599a8-c08c-440d-bc81-4344da4e5128 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:34.247694 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936599a8-c08c-440d-bc81-4344da4e5128 None admin] PATH_INFO: `/v3/users` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:34.248425 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936599a8-c08c-440d-bc81-4344da4e5128 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:34.248729 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936599a8-c08c-440d-bc81-4344da4e5128 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:34.250155 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936599a8-c08c-440d-bc81-4344da4e5128 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:34.273797 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 546/1085] 10.222.0.20 () {56 vars in 1133 bytes} [Thu Sep 22 02:34:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:34.279644 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3b2ee1-4d8b-497f-bbc4-bffb50f446f2 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:34.334843 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:34.387355 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3b2ee1-4d8b-497f-bbc4-bffb50f446f2 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:34.425616 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f771c-1149-4273-bad4-ec9b7d1dc760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:34.426250 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f771c-1149-4273-bad4-ec9b7d1dc760 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:34.426538 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f771c-1149-4273-bad4-ec9b7d1dc760 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:34.426814 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f771c-1149-4273-bad4-ec9b7d1dc760 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:34.501144 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9f771c-1149-4273-bad4-ec9b7d1dc760 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:34.502563 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9f771c-1149-4273-bad4-ec9b7d1dc760 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:34.557581 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:34.611089 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 540/1086] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:34:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 02:34:34.616884 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936599a8-c08c-440d-bc81-4344da4e5128 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:34.678593 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:34.716533 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936599a8-c08c-440d-bc81-4344da4e5128 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:34.750078 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ec77f-633d-4b87-8914-b5b738df31ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:34.750706 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ec77f-633d-4b87-8914-b5b738df31ef None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:34.751012 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ec77f-633d-4b87-8914-b5b738df31ef None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:34.751339 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ec77f-633d-4b87-8914-b5b738df31ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:34.761715 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24ec77f-633d-4b87-8914-b5b738df31ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:34.763186 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24ec77f-633d-4b87-8914-b5b738df31ef None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:34.778868 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-c24ec77f-633d-4b87-8914-b5b738df31ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:34:34.778868 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:34.778868 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:34.778868 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:34.778868 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:34.778868 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:34.778868 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:34.778868 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:34.778868 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:34.778868 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:34.778868 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:34.778868 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:34.778868 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:34:34.778868 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:34:34.778868 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:34:34.778868 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:34:34.779919 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:34.779919 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:34.779919 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:34.779919 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:34.779919 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:34.779919 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:34.779919 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:34.779919 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:34.779919 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:34.779919 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:34.779919 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:34.779919 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:34.779919 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:34.779919 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:34.779919 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:34.779919 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:34.779919 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:34:34.781395 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:34:34.781395 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:34:34.781395 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:34.781395 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 547/1087] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:34:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:34:34.786215 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f771c-1149-4273-bad4-ec9b7d1dc760 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:34.884434 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f771c-1149-4273-bad4-ec9b7d1dc760 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:34.918881 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b40fa-b2e4-4053-b5f5-0d4574a7574f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:34.919487 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b40fa-b2e4-4053-b5f5-0d4574a7574f None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:34.919764 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b40fa-b2e4-4053-b5f5-0d4574a7574f None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:34.920032 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b40fa-b2e4-4053-b5f5-0d4574a7574f None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:34.920673 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8b40fa-b2e4-4053-b5f5-0d4574a7574f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:34.920974 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8b40fa-b2e4-4053-b5f5-0d4574a7574f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:34.922325 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8b40fa-b2e4-4053-b5f5-0d4574a7574f None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:34.933518 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 541/1088] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:34:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:34.939041 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ec77f-633d-4b87-8914-b5b738df31ef None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:35.040133 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ec77f-633d-4b87-8914-b5b738df31ef None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:35.058544 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:35.078000 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14560d9a-f964-40c8-8eb6-c42b552d94a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:35.078633 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14560d9a-f964-40c8-8eb6-c42b552d94a6 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:35.078946 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14560d9a-f964-40c8-8eb6-c42b552d94a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:35.079219 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14560d9a-f964-40c8-8eb6-c42b552d94a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:35.086599 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14560d9a-f964-40c8-8eb6-c42b552d94a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:35.088065 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14560d9a-f964-40c8-8eb6-c42b552d94a6 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:35.099303 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-14560d9a-f964-40c8-8eb6-c42b552d94a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:34:35.099303 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:35.099303 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:35.099303 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:35.099303 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:35.099303 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:35.099303 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:35.099303 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:35.099303 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:35.099303 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:35.099303 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:35.099303 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:35.099303 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:34:35.099303 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:34:35.099303 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:34:35.100375 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:34:35.100375 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:35.100375 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:35.100375 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:35.100375 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:35.100375 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:35.100375 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:35.100375 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:35.100375 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:35.100375 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:35.100375 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:35.100375 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:35.100375 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:35.100375 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:35.100375 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:35.100375 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:35.100375 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:35.106018 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:34:35.106018 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:34:35.106018 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:34:35.106018 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:34:35.106018 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:34:35.106018 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:34:35.106018 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:34:35.106018 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:35.106018 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 548/1089] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:34:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:34:35.106528 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b40fa-b2e4-4053-b5f5-0d4574a7574f None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:35.119334 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:35.164170 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ccb81-715f-4d58-acf1-d3a03ba37203 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:35.164453 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ccb81-715f-4d58-acf1-d3a03ba37203 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:35.164733 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ccb81-715f-4d58-acf1-d3a03ba37203 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:35.166082 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 549/1090] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:34:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:34:35.170994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b705b2-bb27-414b-855b-5ee4a614723e None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:35.171282 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b705b2-bb27-414b-855b-5ee4a614723e None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:35.171564 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b705b2-bb27-414b-855b-5ee4a614723e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:35.190150 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-46b705b2-bb27-414b-855b-5ee4a614723e None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:34:35.191449 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-46b705b2-bb27-414b-855b-5ee4a614723e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:34:35.211179 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b40fa-b2e4-4053-b5f5-0d4574a7574f None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:35.245933 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c157488-4dd9-431a-afc1-c082a48f9a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:35.246543 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c157488-4dd9-431a-afc1-c082a48f9a2b None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:35.246833 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c157488-4dd9-431a-afc1-c082a48f9a2b None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:35.247125 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c157488-4dd9-431a-afc1-c082a48f9a2b None admin] PATH_INFO: `/v3/projects` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:35.247816 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c157488-4dd9-431a-afc1-c082a48f9a2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:35.248127 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c157488-4dd9-431a-afc1-c082a48f9a2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:35.249547 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c157488-4dd9-431a-afc1-c082a48f9a2b None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:35.259142 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 550/1091] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:34:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 02:34:35.266922 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 542/1092] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Sep 22 02:34:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:35.268075 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b705b2-bb27-414b-855b-5ee4a614723e None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:35.366906 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b705b2-bb27-414b-855b-5ee4a614723e None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:35.373983 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:35.401729 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b7034b-f675-425b-9739-29eedec3eb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:35.402341 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7034b-f675-425b-9739-29eedec3eb9b None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:35.402622 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7034b-f675-425b-9739-29eedec3eb9b None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:35.402923 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7034b-f675-425b-9739-29eedec3eb9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:35.403653 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b7034b-f675-425b-9739-29eedec3eb9b None admin] RBAC: Adding query filter params (role.id=daf8ca6bfd534c00806f8cdfeb3bac5d, scope.project.id=2a792f8dc45145d687ddd58c94d44b5e, user.id=a64ed6c8e15d479887ab1dc59997d715) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:35.403964 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b7034b-f675-425b-9739-29eedec3eb9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:35.405378 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b7034b-f675-425b-9739-29eedec3eb9b None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:35.411075 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 551/1093] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:34:35 2022] GET /identity/v3/role_assignments?user.id=a64ed6c8e15d479887ab1dc59997d715&scope.project.id=2a792f8dc45145d687ddd58c94d44b5e&role.id=daf8ca6bfd534c00806f8cdfeb3bac5d => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:35.530371 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c157488-4dd9-431a-afc1-c082a48f9a2b None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:35.604897 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:35.640149 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c157488-4dd9-431a-afc1-c082a48f9a2b None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:35.676822 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357481a-b311-4309-ae9e-941250e36b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:35.677462 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357481a-b311-4309-ae9e-941250e36b4e None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:35.677759 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357481a-b311-4309-ae9e-941250e36b4e None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:35.678013 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357481a-b311-4309-ae9e-941250e36b4e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:35.688566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9357481a-b311-4309-ae9e-941250e36b4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:34:35.693514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-9357481a-b311-4309-ae9e-941250e36b4e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:34:35.693514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:35.693514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:35.693514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:35.693514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:35.693514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:35.693514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:35.693514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:35.693514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:35.693514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:35.693514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:35.693514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:35.693514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:34:35.693514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:34:35.693514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:34:35.693514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:34:35.694494 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:34:35.694494 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:34:35.694494 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:34:35.694494 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:34:35.694494 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:35.694494 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:35.694494 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:35.694494 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:35.694494 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:35.694494 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:35.694494 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:35.694494 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:35.694494 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:35.694494 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:35.694494 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:35.694494 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:35.695400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:35.695400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:35.695400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:35.695400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:35.695400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:34:35.695400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:34:35.695400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:34:35.695400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:34:35.695400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:34:35.695400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:34:35.695400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:34:35.695400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:34:35.695400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 543/1094] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Sep 22 02:34:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:34:35.700952 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b7034b-f675-425b-9739-29eedec3eb9b None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:35.806700 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b7034b-f675-425b-9739-29eedec3eb9b None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:35.841242 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1427fd30-f6b6-489d-b0ce-2fd40d8a1f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:35.841854 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1427fd30-f6b6-489d-b0ce-2fd40d8a1f88 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:35.842141 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1427fd30-f6b6-489d-b0ce-2fd40d8a1f88 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:35.842414 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1427fd30-f6b6-489d-b0ce-2fd40d8a1f88 None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:35.843101 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1427fd30-f6b6-489d-b0ce-2fd40d8a1f88 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:35.843417 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1427fd30-f6b6-489d-b0ce-2fd40d8a1f88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:35.844756 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1427fd30-f6b6-489d-b0ce-2fd40d8a1f88 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:35.854442 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 552/1095] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:34:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:35.859683 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357481a-b311-4309-ae9e-941250e36b4e None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:35.968566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357481a-b311-4309-ae9e-941250e36b4e None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:35.999732 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:36.006199 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d141aa-d88c-44ce-9f72-a94ef9b49757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:36.006896 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d141aa-d88c-44ce-9f72-a94ef9b49757 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:36.007190 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d141aa-d88c-44ce-9f72-a94ef9b49757 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:36.007464 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d141aa-d88c-44ce-9f72-a94ef9b49757 None admin] PATH_INFO: `/v3/users/c02156f0a19a4996a77c3837d9876fd9` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:36.008619 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d141aa-d88c-44ce-9f72-a94ef9b49757 None admin] RBAC: Authorizing `identity:get_user(user_id=c02156f0a19a4996a77c3837d9876fd9)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:36.010066 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d141aa-d88c-44ce-9f72-a94ef9b49757 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:36.012049 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 544/1096] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:34:35 2022] GET /identity/v3/users/c02156f0a19a4996a77c3837d9876fd9 => generated 285 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:36.021823 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1427fd30-f6b6-489d-b0ce-2fd40d8a1f88 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:36.057880 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:36.127209 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1427fd30-f6b6-489d-b0ce-2fd40d8a1f88 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:36.162201 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea297a0c-b737-4e54-8654-550551148cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:36.162862 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea297a0c-b737-4e54-8654-550551148cb1 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:36.163149 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea297a0c-b737-4e54-8654-550551148cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:36.163422 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea297a0c-b737-4e54-8654-550551148cb1 None admin] PATH_INFO: `/v3/projects/fd36fd0411884b52b3319ef42551a7ef` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:36.164541 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea297a0c-b737-4e54-8654-550551148cb1 None admin] RBAC: Authorizing `identity:get_project(project_id=fd36fd0411884b52b3319ef42551a7ef)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:36.165970 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea297a0c-b737-4e54-8654-550551148cb1 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:36.168454 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 553/1097] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:34:36 2022] GET /identity/v3/projects/fd36fd0411884b52b3319ef42551a7ef => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:36.173846 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d141aa-d88c-44ce-9f72-a94ef9b49757 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:36.278343 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d141aa-d88c-44ce-9f72-a94ef9b49757 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:36.318707 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99770f2-d758-4e06-b1d4-e6f17a82fcae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:36.319401 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99770f2-d758-4e06-b1d4-e6f17a82fcae None admin] REQUEST_METHOD: `PUT` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:36.319681 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99770f2-d758-4e06-b1d4-e6f17a82fcae None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:36.319956 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99770f2-d758-4e06-b1d4-e6f17a82fcae None admin] PATH_INFO: `/v3/projects/fd36fd0411884b52b3319ef42551a7ef/users/c02156f0a19a4996a77c3837d9876fd9/roles/31a687548f55493b8ea666fe7e350303` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:36.321838 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99770f2-d758-4e06-b1d4-e6f17a82fcae None admin] RBAC: Authorizing `identity:create_grant(project_id=fd36fd0411884b52b3319ef42551a7ef, user_id=c02156f0a19a4996a77c3837d9876fd9, role_id=31a687548f55493b8ea666fe7e350303)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:36.323325 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99770f2-d758-4e06-b1d4-e6f17a82fcae None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:36.336377 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 545/1098] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Sep 22 02:34:36 2022] PUT /identity/v3/projects/fd36fd0411884b52b3319ef42551a7ef/users/c02156f0a19a4996a77c3837d9876fd9/roles/31a687548f55493b8ea666fe7e350303 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 02:34:38.469856 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd044e48-6ddb-4f2c-a9ca-77ee4ce21a92 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:38.470432 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd044e48-6ddb-4f2c-a9ca-77ee4ce21a92 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:38.470432 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd044e48-6ddb-4f2c-a9ca-77ee4ce21a92 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:38.472346 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 554/1099] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:34:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:34:38.477813 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3e2ee-f5d3-403d-b9f4-f86793fb2894 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:38.478261 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3e2ee-f5d3-403d-b9f4-f86793fb2894 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:38.478376 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3e2ee-f5d3-403d-b9f4-f86793fb2894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:38.497329 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-e5e3e2ee-f5d3-403d-b9f4-f86793fb2894 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:34:38.498520 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-e5e3e2ee-f5d3-403d-b9f4-f86793fb2894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:34:38.551723 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:38.567171 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 546/1100] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:34:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:34:38.842160 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd044e48-6ddb-4f2c-a9ca-77ee4ce21a92 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:38.938361 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:38.943140 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd044e48-6ddb-4f2c-a9ca-77ee4ce21a92 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:38.976908 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ba3bc-b129-4afb-a42d-4198f6404170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:38.977544 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ba3bc-b129-4afb-a42d-4198f6404170 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:38.977829 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ba3bc-b129-4afb-a42d-4198f6404170 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:38.978109 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ba3bc-b129-4afb-a42d-4198f6404170 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:38.988973 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ba3bc-b129-4afb-a42d-4198f6404170 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:34:38.993883 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-5f2ba3bc-b129-4afb-a42d-4198f6404170 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:34:38.993883 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:34:38.993883 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:34:38.993883 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:34:38.993883 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:34:38.993883 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:34:38.993883 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:34:38.993883 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:34:38.993883 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:34:38.993883 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:34:38.993883 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:34:38.993883 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:34:38.993883 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:34:38.993883 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:34:38.993883 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:34:38.993883 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:34:38.994994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:34:38.994994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:34:38.994994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:34:38.994994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:34:38.994994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:34:38.994994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:34:38.994994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:34:38.994994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:34:38.994994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:34:38.994994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:34:38.994994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:34:38.994994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:34:38.994994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:34:38.994994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:34:38.994994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:34:38.994994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:34:38.995847 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:34:38.995847 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:34:38.995847 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:34:38.995847 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:34:38.995847 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:34:38.995847 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:34:38.995847 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:34:38.995847 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:34:38.995847 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:34:38.995847 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:34:38.995847 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:34:38.995847 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:34:38.995847 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 555/1101] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Sep 22 02:34:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:34:39.001369 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e3e2ee-f5d3-403d-b9f4-f86793fb2894 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:39.100236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e3e2ee-f5d3-403d-b9f4-f86793fb2894 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:39.134100 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d1d519-e558-410b-9303-e9f8d35738b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:39.134709 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1d519-e558-410b-9303-e9f8d35738b3 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:39.135021 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1d519-e558-410b-9303-e9f8d35738b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:39.135300 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1d519-e558-410b-9303-e9f8d35738b3 None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:39.135966 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d1d519-e558-410b-9303-e9f8d35738b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:39.136274 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d1d519-e558-410b-9303-e9f8d35738b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:39.137636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d1d519-e558-410b-9303-e9f8d35738b3 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:39.145880 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 547/1102] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:34:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:39.152167 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ba3bc-b129-4afb-a42d-4198f6404170 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:39.250072 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ba3bc-b129-4afb-a42d-4198f6404170 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:39.284609 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12145e95-7c87-42b0-b442-47e07063631e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:39.285241 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12145e95-7c87-42b0-b442-47e07063631e None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:39.285526 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12145e95-7c87-42b0-b442-47e07063631e None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:39.285801 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12145e95-7c87-42b0-b442-47e07063631e None admin] PATH_INFO: `/v3/users/c02156f0a19a4996a77c3837d9876fd9` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:39.286941 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12145e95-7c87-42b0-b442-47e07063631e None admin] RBAC: Authorizing `identity:get_user(user_id=c02156f0a19a4996a77c3837d9876fd9)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:39.288377 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12145e95-7c87-42b0-b442-47e07063631e None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:39.289961 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 556/1103] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:34:39 2022] GET /identity/v3/users/c02156f0a19a4996a77c3837d9876fd9 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:39.295845 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d1d519-e558-410b-9303-e9f8d35738b3 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:39.344337 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:39.394634 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d1d519-e558-410b-9303-e9f8d35738b3 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:39.429840 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757a729-9a92-4f8b-8f67-97850d73e39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:39.430465 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757a729-9a92-4f8b-8f67-97850d73e39b None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:39.430748 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757a729-9a92-4f8b-8f67-97850d73e39b None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:39.431054 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757a729-9a92-4f8b-8f67-97850d73e39b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:39.498918 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757a729-9a92-4f8b-8f67-97850d73e39b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:39.500368 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757a729-9a92-4f8b-8f67-97850d73e39b None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:39.549071 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:39.600128 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 548/1104] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:34:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 02:34:39.606366 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12145e95-7c87-42b0-b442-47e07063631e None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:39.637595 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:39.710423 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12145e95-7c87-42b0-b442-47e07063631e None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:39.743047 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8dc3c9-40e3-4c9a-ac97-01fa5af012de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:39.743667 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8dc3c9-40e3-4c9a-ac97-01fa5af012de None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:39.744045 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8dc3c9-40e3-4c9a-ac97-01fa5af012de None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:39.744213 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8dc3c9-40e3-4c9a-ac97-01fa5af012de None admin] PATH_INFO: `/v3/projects/fd36fd0411884b52b3319ef42551a7ef` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:39.745312 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8dc3c9-40e3-4c9a-ac97-01fa5af012de None admin] RBAC: Authorizing `identity:get_project(project_id=fd36fd0411884b52b3319ef42551a7ef)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:39.746729 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8dc3c9-40e3-4c9a-ac97-01fa5af012de None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:39.748370 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 557/1105] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:34:39 2022] GET /identity/v3/projects/fd36fd0411884b52b3319ef42551a7ef => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:34:39.755231 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757a729-9a92-4f8b-8f67-97850d73e39b None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:39.851177 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757a729-9a92-4f8b-8f67-97850d73e39b None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:39.884786 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097291f-e1a5-4957-b5f8-d96e57291a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:39.885385 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097291f-e1a5-4957-b5f8-d96e57291a35 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:39.885670 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097291f-e1a5-4957-b5f8-d96e57291a35 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:39.885946 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097291f-e1a5-4957-b5f8-d96e57291a35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:39.886674 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6097291f-e1a5-4957-b5f8-d96e57291a35 None admin] RBAC: Adding query filter params (role.id=31a687548f55493b8ea666fe7e350303, scope.project.id=fd36fd0411884b52b3319ef42551a7ef, user.id=c02156f0a19a4996a77c3837d9876fd9) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:39.887016 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6097291f-e1a5-4957-b5f8-d96e57291a35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:39.888445 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6097291f-e1a5-4957-b5f8-d96e57291a35 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:39.894084 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 549/1106] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:34:39 2022] GET /identity/v3/role_assignments?user.id=c02156f0a19a4996a77c3837d9876fd9&scope.project.id=fd36fd0411884b52b3319ef42551a7ef&role.id=31a687548f55493b8ea666fe7e350303 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:42.070625 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc7305-acca-42a6-b80f-94ce45f0137d None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:42.071182 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc7305-acca-42a6-b80f-94ce45f0137d None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:42.071182 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc7305-acca-42a6-b80f-94ce45f0137d None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:42.072540 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 558/1107] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:34:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:34:42.077806 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83013935-e674-41f5-a909-e1f728f5afd9 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:42.078181 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83013935-e674-41f5-a909-e1f728f5afd9 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:42.078352 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83013935-e674-41f5-a909-e1f728f5afd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:42.096201 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-83013935-e674-41f5-a909-e1f728f5afd9 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:34:42.097376 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-83013935-e674-41f5-a909-e1f728f5afd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:34:42.106407 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:42.162598 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 550/1108] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:34:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:34:42.435980 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bc7305-acca-42a6-b80f-94ce45f0137d None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:42.529245 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:42.536766 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bc7305-acca-42a6-b80f-94ce45f0137d None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:42.570529 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1069513-3381-4b1b-bfb0-961418a38954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:42.571176 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1069513-3381-4b1b-bfb0-961418a38954 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:42.571456 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1069513-3381-4b1b-bfb0-961418a38954 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:42.571727 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1069513-3381-4b1b-bfb0-961418a38954 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:42.572826 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1069513-3381-4b1b-bfb0-961418a38954 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:42.574254 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1069513-3381-4b1b-bfb0-961418a38954 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:42.575851 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 559/1109] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:34:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:42.582272 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83013935-e674-41f5-a909-e1f728f5afd9 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:42.682067 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83013935-e674-41f5-a909-e1f728f5afd9 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:42.715733 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99102553-9ab4-4824-ba4e-c5bdfb3c32bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:42.716283 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99102553-9ab4-4824-ba4e-c5bdfb3c32bb None admin] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:42.716564 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99102553-9ab4-4824-ba4e-c5bdfb3c32bb None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:42.716839 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99102553-9ab4-4824-ba4e-c5bdfb3c32bb None admin] PATH_INFO: `/v3/users` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:42.717497 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99102553-9ab4-4824-ba4e-c5bdfb3c32bb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:42.718957 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99102553-9ab4-4824-ba4e-c5bdfb3c32bb None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:42.735742 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:42.749286 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 551/1110] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Sep 22 02:34:42 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:34:44.951152 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41546f9-a417-43c0-a4f6-29cc42e19b7d None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:44.951695 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41546f9-a417-43c0-a4f6-29cc42e19b7d None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:44.951695 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41546f9-a417-43c0-a4f6-29cc42e19b7d None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:44.953046 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 560/1111] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:34:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:34:44.958243 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec4ea3-6885-45dc-adf8-91788a5535a3 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:44.958658 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec4ea3-6885-45dc-adf8-91788a5535a3 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:44.958817 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec4ea3-6885-45dc-adf8-91788a5535a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:44.977106 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-c6ec4ea3-6885-45dc-adf8-91788a5535a3 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:34:44.978278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-c6ec4ea3-6885-45dc-adf8-91788a5535a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:34:45.045049 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 552/1112] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:34:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:34:45.310508 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41546f9-a417-43c0-a4f6-29cc42e19b7d None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:45.411373 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41546f9-a417-43c0-a4f6-29cc42e19b7d None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:45.444437 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bd550-0e79-444a-8216-d494d9eb9c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:45.445062 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bd550-0e79-444a-8216-d494d9eb9c05 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:45.445342 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bd550-0e79-444a-8216-d494d9eb9c05 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:45.445614 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bd550-0e79-444a-8216-d494d9eb9c05 None admin] PATH_INFO: `/v3/roles/adec3004b56243fa923bd9f0082a2728` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:45.447013 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bd550-0e79-444a-8216-d494d9eb9c05 None admin] RBAC: Authorizing `identity:get_role(role_id=adec3004b56243fa923bd9f0082a2728)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:45.448364 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bd550-0e79-444a-8216-d494d9eb9c05 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:45.449672 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 561/1113] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:34:45 2022] GET /identity/v3/roles/adec3004b56243fa923bd9f0082a2728 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:45.455566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec4ea3-6885-45dc-adf8-91788a5535a3 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:45.553828 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec4ea3-6885-45dc-adf8-91788a5535a3 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:45.581608 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:45.587617 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58741b4-5d9c-42e9-ad56-a019eb9b9506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:45.588261 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58741b4-5d9c-42e9-ad56-a019eb9b9506 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:45.588545 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58741b4-5d9c-42e9-ad56-a019eb9b9506 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:45.588821 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58741b4-5d9c-42e9-ad56-a019eb9b9506 None admin] PATH_INFO: `/v3/users/c6856315c5cd4e7f9b76c82e6b20fb6b` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:45.603311 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58741b4-5d9c-42e9-ad56-a019eb9b9506 None admin] RBAC: Authorizing `identity:get_user(user_id=c6856315c5cd4e7f9b76c82e6b20fb6b)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:45.604761 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58741b4-5d9c-42e9-ad56-a019eb9b9506 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:45.606407 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 553/1114] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:34:45 2022] GET /identity/v3/users/c6856315c5cd4e7f9b76c82e6b20fb6b => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:45.612843 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bd550-0e79-444a-8216-d494d9eb9c05 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:45.643454 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:45.711521 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bd550-0e79-444a-8216-d494d9eb9c05 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:45.744845 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4f02e-5dfb-41e9-9b08-b63148b7f00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:45.745455 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4f02e-5dfb-41e9-9b08-b63148b7f00d None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:45.745741 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4f02e-5dfb-41e9-9b08-b63148b7f00d None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:45.746018 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4f02e-5dfb-41e9-9b08-b63148b7f00d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:45.811576 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4f02e-5dfb-41e9-9b08-b63148b7f00d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:45.813026 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4f02e-5dfb-41e9-9b08-b63148b7f00d None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:45.841936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:45.909717 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 562/1115] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:34:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 02:34:45.915719 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58741b4-5d9c-42e9-ad56-a019eb9b9506 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:46.019699 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58741b4-5d9c-42e9-ad56-a019eb9b9506 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:46.056053 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862690bd-f6c6-457f-9c2d-1fbc44090880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:46.056683 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862690bd-f6c6-457f-9c2d-1fbc44090880 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:46.056970 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862690bd-f6c6-457f-9c2d-1fbc44090880 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:46.057247 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862690bd-f6c6-457f-9c2d-1fbc44090880 None admin] PATH_INFO: `/v3/projects/fd36fd0411884b52b3319ef42551a7ef` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:46.058357 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862690bd-f6c6-457f-9c2d-1fbc44090880 None admin] RBAC: Authorizing `identity:get_project(project_id=fd36fd0411884b52b3319ef42551a7ef)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:46.059822 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862690bd-f6c6-457f-9c2d-1fbc44090880 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:46.061454 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 554/1116] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:34:45 2022] GET /identity/v3/projects/fd36fd0411884b52b3319ef42551a7ef => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:46.068327 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4f02e-5dfb-41e9-9b08-b63148b7f00d None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:46.168994 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4f02e-5dfb-41e9-9b08-b63148b7f00d None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:46.197845 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:46.203174 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36dc4c-617c-422a-b39c-8ce648a1c35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:46.203777 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36dc4c-617c-422a-b39c-8ce648a1c35e None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:46.204055 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36dc4c-617c-422a-b39c-8ce648a1c35e None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:46.204326 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36dc4c-617c-422a-b39c-8ce648a1c35e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:46.205043 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d36dc4c-617c-422a-b39c-8ce648a1c35e None admin] RBAC: Adding query filter params (role.id=adec3004b56243fa923bd9f0082a2728, scope.project.id=fd36fd0411884b52b3319ef42551a7ef, user.id=c6856315c5cd4e7f9b76c82e6b20fb6b) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:46.205348 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d36dc4c-617c-422a-b39c-8ce648a1c35e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:46.206762 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d36dc4c-617c-422a-b39c-8ce648a1c35e None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:46.212100 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 563/1117] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:34:46 2022] GET /identity/v3/role_assignments?user.id=c6856315c5cd4e7f9b76c82e6b20fb6b&scope.project.id=fd36fd0411884b52b3319ef42551a7ef&role.id=adec3004b56243fa923bd9f0082a2728 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:48.336272 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358df3e-70a3-473f-bb89-8c01fbd2cdaf None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:48.336799 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358df3e-70a3-473f-bb89-8c01fbd2cdaf None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:48.336799 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358df3e-70a3-473f-bb89-8c01fbd2cdaf None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:48.338141 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 555/1118] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:34:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:34:48.343237 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7885c2ae-f580-433a-bfc9-67946131a78c None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:48.343664 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7885c2ae-f580-433a-bfc9-67946131a78c None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:48.343773 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7885c2ae-f580-433a-bfc9-67946131a78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:48.361928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-7885c2ae-f580-433a-bfc9-67946131a78c None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:34:48.363128 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-7885c2ae-f580-433a-bfc9-67946131a78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:34:48.428842 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 564/1119] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:34:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:34:48.697296 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5358df3e-70a3-473f-bb89-8c01fbd2cdaf None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:48.755020 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:48.808676 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5358df3e-70a3-473f-bb89-8c01fbd2cdaf None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:48.841050 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d730ca44-5f9b-4b44-bb15-1180ce55b08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:48.841675 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730ca44-5f9b-4b44-bb15-1180ce55b08a None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:48.841954 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730ca44-5f9b-4b44-bb15-1180ce55b08a None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:48.842224 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730ca44-5f9b-4b44-bb15-1180ce55b08a None admin] PATH_INFO: `/v3/roles/adec3004b56243fa923bd9f0082a2728` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:48.843624 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d730ca44-5f9b-4b44-bb15-1180ce55b08a None admin] RBAC: Authorizing `identity:get_role(role_id=adec3004b56243fa923bd9f0082a2728)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:48.844980 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d730ca44-5f9b-4b44-bb15-1180ce55b08a None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:48.846929 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 556/1120] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:34:48 2022] GET /identity/v3/roles/adec3004b56243fa923bd9f0082a2728 => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:48.852810 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7885c2ae-f580-433a-bfc9-67946131a78c None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:48.951765 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7885c2ae-f580-433a-bfc9-67946131a78c None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:48.977977 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:48.986612 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96823d9-b43c-4530-977b-eec4ea4da122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:48.987287 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96823d9-b43c-4530-977b-eec4ea4da122 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:48.987574 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96823d9-b43c-4530-977b-eec4ea4da122 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:48.987855 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96823d9-b43c-4530-977b-eec4ea4da122 None admin] PATH_INFO: `/v3/users/c6856315c5cd4e7f9b76c82e6b20fb6b` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:48.988979 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96823d9-b43c-4530-977b-eec4ea4da122 None admin] RBAC: Authorizing `identity:get_user(user_id=c6856315c5cd4e7f9b76c82e6b20fb6b)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:48.990422 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96823d9-b43c-4530-977b-eec4ea4da122 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:48.992075 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 565/1121] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:34:48 2022] GET /identity/v3/users/c6856315c5cd4e7f9b76c82e6b20fb6b => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:48.997979 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d730ca44-5f9b-4b44-bb15-1180ce55b08a None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:49.098296 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d730ca44-5f9b-4b44-bb15-1180ce55b08a None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:49.121304 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:49.133047 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f2e9b-632c-41b2-9299-df2d088da8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:49.133679 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f2e9b-632c-41b2-9299-df2d088da8f2 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:49.133961 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f2e9b-632c-41b2-9299-df2d088da8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:49.134239 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f2e9b-632c-41b2-9299-df2d088da8f2 None admin] PATH_INFO: `/v3/projects/fd36fd0411884b52b3319ef42551a7ef` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:49.135389 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f2e9b-632c-41b2-9299-df2d088da8f2 None admin] RBAC: Authorizing `identity:get_project(project_id=fd36fd0411884b52b3319ef42551a7ef)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:49.136828 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f2e9b-632c-41b2-9299-df2d088da8f2 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:49.138486 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 557/1122] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:34:48 2022] GET /identity/v3/projects/fd36fd0411884b52b3319ef42551a7ef => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:49.145212 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96823d9-b43c-4530-977b-eec4ea4da122 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:49.241545 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96823d9-b43c-4530-977b-eec4ea4da122 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:49.275583 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c2870-ed29-4b2f-8cd9-d6b835d5923d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:49.276237 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c2870-ed29-4b2f-8cd9-d6b835d5923d None admin] REQUEST_METHOD: `PUT` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:49.276517 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c2870-ed29-4b2f-8cd9-d6b835d5923d None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:49.276792 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c2870-ed29-4b2f-8cd9-d6b835d5923d None admin] PATH_INFO: `/v3/projects/fd36fd0411884b52b3319ef42551a7ef/users/c6856315c5cd4e7f9b76c82e6b20fb6b/roles/adec3004b56243fa923bd9f0082a2728` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:49.278906 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c2870-ed29-4b2f-8cd9-d6b835d5923d None admin] RBAC: Authorizing `identity:create_grant(project_id=fd36fd0411884b52b3319ef42551a7ef, user_id=c6856315c5cd4e7f9b76c82e6b20fb6b, role_id=adec3004b56243fa923bd9f0082a2728)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:49.280325 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c2870-ed29-4b2f-8cd9-d6b835d5923d None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:49.292735 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 566/1123] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Sep 22 02:34:49 2022] PUT /identity/v3/projects/fd36fd0411884b52b3319ef42551a7ef/users/c6856315c5cd4e7f9b76c82e6b20fb6b/roles/adec3004b56243fa923bd9f0082a2728 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 02:34:51.475111 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e73c0-fb0f-49b0-86a4-c5ec50f7cab0 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:51.475651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e73c0-fb0f-49b0-86a4-c5ec50f7cab0 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:51.475742 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e73c0-fb0f-49b0-86a4-c5ec50f7cab0 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:51.477078 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 558/1124] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:34:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:34:51.482437 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a7d58-bcb8-4712-8131-00dcda2ae877 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:51.482867 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a7d58-bcb8-4712-8131-00dcda2ae877 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:51.483034 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a7d58-bcb8-4712-8131-00dcda2ae877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:51.501507 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-fb8a7d58-bcb8-4712-8131-00dcda2ae877 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:34:51.502710 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-fb8a7d58-bcb8-4712-8131-00dcda2ae877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:34:51.551532 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:51.570200 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 567/1125] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:34:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:34:51.850849 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e73c0-fb0f-49b0-86a4-c5ec50f7cab0 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:51.950081 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e73c0-fb0f-49b0-86a4-c5ec50f7cab0 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:51.984489 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b66f61-0946-4b95-aded-2fd2a4fb2902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:51.985125 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b66f61-0946-4b95-aded-2fd2a4fb2902 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:51.985428 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b66f61-0946-4b95-aded-2fd2a4fb2902 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:51.985719 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b66f61-0946-4b95-aded-2fd2a4fb2902 None admin] PATH_INFO: `/v3/roles/adec3004b56243fa923bd9f0082a2728` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:51.987173 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b66f61-0946-4b95-aded-2fd2a4fb2902 None admin] RBAC: Authorizing `identity:get_role(role_id=adec3004b56243fa923bd9f0082a2728)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:51.988545 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b66f61-0946-4b95-aded-2fd2a4fb2902 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:51.989870 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 559/1126] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:34:51 2022] GET /identity/v3/roles/adec3004b56243fa923bd9f0082a2728 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:51.995874 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8a7d58-bcb8-4712-8131-00dcda2ae877 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:52.093833 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8a7d58-bcb8-4712-8131-00dcda2ae877 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:52.128997 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f02d7b0-9fd7-4f8e-a6ec-61a320790498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:52.129547 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02d7b0-9fd7-4f8e-a6ec-61a320790498 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:52.129831 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02d7b0-9fd7-4f8e-a6ec-61a320790498 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:52.130104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02d7b0-9fd7-4f8e-a6ec-61a320790498 None admin] PATH_INFO: `/v3/users/c6856315c5cd4e7f9b76c82e6b20fb6b` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:52.131543 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f02d7b0-9fd7-4f8e-a6ec-61a320790498 None admin] RBAC: Authorizing `identity:get_user(user_id=c6856315c5cd4e7f9b76c82e6b20fb6b)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:52.132686 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f02d7b0-9fd7-4f8e-a6ec-61a320790498 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:52.134266 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 568/1127] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:34:51 2022] GET /identity/v3/users/c6856315c5cd4e7f9b76c82e6b20fb6b => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:52.140262 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b66f61-0946-4b95-aded-2fd2a4fb2902 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:52.199104 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:52.239963 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b66f61-0946-4b95-aded-2fd2a4fb2902 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:52.277954 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a6888-ed65-4190-a8f9-4d7ced99a9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:52.278585 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a6888-ed65-4190-a8f9-4d7ced99a9f4 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:52.278904 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a6888-ed65-4190-a8f9-4d7ced99a9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:52.279183 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a6888-ed65-4190-a8f9-4d7ced99a9f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:52.345824 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302a6888-ed65-4190-a8f9-4d7ced99a9f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:52.347303 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302a6888-ed65-4190-a8f9-4d7ced99a9f4 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:52.406139 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:52.447019 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 560/1128] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:34:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 02:34:52.452972 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f02d7b0-9fd7-4f8e-a6ec-61a320790498 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:52.506150 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:52.554383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f02d7b0-9fd7-4f8e-a6ec-61a320790498 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:52.587522 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8ce6b-71c5-4e89-9c58-9411e72bcfe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:52.588150 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8ce6b-71c5-4e89-9c58-9411e72bcfe8 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:52.588433 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8ce6b-71c5-4e89-9c58-9411e72bcfe8 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:52.588709 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8ce6b-71c5-4e89-9c58-9411e72bcfe8 None admin] PATH_INFO: `/v3/projects/fd36fd0411884b52b3319ef42551a7ef` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:52.589818 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b8ce6b-71c5-4e89-9c58-9411e72bcfe8 None admin] RBAC: Authorizing `identity:get_project(project_id=fd36fd0411884b52b3319ef42551a7ef)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:52.591284 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b8ce6b-71c5-4e89-9c58-9411e72bcfe8 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:52.592905 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 569/1129] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:34:52 2022] GET /identity/v3/projects/fd36fd0411884b52b3319ef42551a7ef => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:52.599191 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a6888-ed65-4190-a8f9-4d7ced99a9f4 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:52.697024 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a6888-ed65-4190-a8f9-4d7ced99a9f4 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:52.729580 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92faf68-94c7-4e11-998c-d95d038664e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:52.730178 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92faf68-94c7-4e11-998c-d95d038664e8 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:52.730463 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92faf68-94c7-4e11-998c-d95d038664e8 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:52.730741 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92faf68-94c7-4e11-998c-d95d038664e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:52.731499 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92faf68-94c7-4e11-998c-d95d038664e8 None admin] RBAC: Adding query filter params (role.id=adec3004b56243fa923bd9f0082a2728, scope.project.id=fd36fd0411884b52b3319ef42551a7ef, user.id=c6856315c5cd4e7f9b76c82e6b20fb6b) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:34:52.731815 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92faf68-94c7-4e11-998c-d95d038664e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:52.733239 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92faf68-94c7-4e11-998c-d95d038664e8 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:52.739728 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 561/1130] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:34:52 2022] GET /identity/v3/role_assignments?user.id=c6856315c5cd4e7f9b76c82e6b20fb6b&scope.project.id=fd36fd0411884b52b3319ef42551a7ef&role.id=adec3004b56243fa923bd9f0082a2728 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:54.914241 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30726bc9-1557-4f07-adf9-d1923083f39c None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:54.914875 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30726bc9-1557-4f07-adf9-d1923083f39c None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:54.914875 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30726bc9-1557-4f07-adf9-d1923083f39c None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:54.916296 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 570/1131] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:34:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:34:54.921534 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f373004-2a18-483f-b913-75d92e3bceed None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:54.922001 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f373004-2a18-483f-b913-75d92e3bceed None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:54.922100 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f373004-2a18-483f-b913-75d92e3bceed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:54.942421 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-7f373004-2a18-483f-b913-75d92e3bceed None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:34:54.943663 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-7f373004-2a18-483f-b913-75d92e3bceed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:34:54.961093 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:55.009824 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 562/1132] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:34:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 02:34:55.308280 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30726bc9-1557-4f07-adf9-d1923083f39c None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:55.410995 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30726bc9-1557-4f07-adf9-d1923083f39c None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:55.430275 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:55.444359 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597e2ce-668b-4830-bef3-65a4a3f6e1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:55.444988 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597e2ce-668b-4830-bef3-65a4a3f6e1f1 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:55.445270 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597e2ce-668b-4830-bef3-65a4a3f6e1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:55.445546 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597e2ce-668b-4830-bef3-65a4a3f6e1f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:55.446658 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d597e2ce-668b-4830-bef3-65a4a3f6e1f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:55.448119 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d597e2ce-668b-4830-bef3-65a4a3f6e1f1 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:55.449681 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 571/1133] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:34:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:55.456106 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f373004-2a18-483f-b913-75d92e3bceed None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:55.555938 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f373004-2a18-483f-b913-75d92e3bceed None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:55.589267 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea28c41-e018-4ff1-86f9-e1c53ff248ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:55.589877 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea28c41-e018-4ff1-86f9-e1c53ff248ca None admin] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:55.590160 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea28c41-e018-4ff1-86f9-e1c53ff248ca None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:55.590435 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea28c41-e018-4ff1-86f9-e1c53ff248ca None admin] PATH_INFO: `/v3/projects` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:55.591109 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea28c41-e018-4ff1-86f9-e1c53ff248ca None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:55.592549 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea28c41-e018-4ff1-86f9-e1c53ff248ca None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:55.619388 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 563/1134] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Sep 22 02:34:55 2022] POST /identity/v3/projects => generated 311 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:34:57.824221 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beced8bc-ebfe-47b5-92e3-0d0bc9ae35bf None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:57.824783 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beced8bc-ebfe-47b5-92e3-0d0bc9ae35bf None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:57.824783 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beced8bc-ebfe-47b5-92e3-0d0bc9ae35bf None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:57.826123 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 572/1135] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:34:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:34:57.831456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d23b2-2438-459c-851e-02a3a056faf8 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:57.831894 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d23b2-2438-459c-851e-02a3a056faf8 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:57.832017 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d23b2-2438-459c-851e-02a3a056faf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:57.844928 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:57.851317 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-ec1d23b2-2438-459c-851e-02a3a056faf8 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:34:57.852525 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-ec1d23b2-2438-459c-851e-02a3a056faf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:34:57.918674 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 564/1136] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:34:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:34:58.192316 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beced8bc-ebfe-47b5-92e3-0d0bc9ae35bf None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:58.291919 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beced8bc-ebfe-47b5-92e3-0d0bc9ae35bf None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:58.325875 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371abd75-c9b7-498f-b097-03e5cc60e79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:58.326503 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371abd75-c9b7-498f-b097-03e5cc60e79c None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:58.326783 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371abd75-c9b7-498f-b097-03e5cc60e79c None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:58.327088 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371abd75-c9b7-498f-b097-03e5cc60e79c None admin] PATH_INFO: `/v3/domains/default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:58.328198 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371abd75-c9b7-498f-b097-03e5cc60e79c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:58.330284 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371abd75-c9b7-498f-b097-03e5cc60e79c None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:58.331903 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 573/1137] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:34:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:34:58.337817 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d23b2-2438-459c-851e-02a3a056faf8 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:34:58.434279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d23b2-2438-459c-851e-02a3a056faf8 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:34:58.475809 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290630d-30bc-4bca-bffd-4dae9edd529e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:34:58.476346 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290630d-30bc-4bca-bffd-4dae9edd529e None admin] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:34:58.476633 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290630d-30bc-4bca-bffd-4dae9edd529e None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:34:58.476912 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290630d-30bc-4bca-bffd-4dae9edd529e None admin] PATH_INFO: `/v3/users` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:34:58.477578 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b290630d-30bc-4bca-bffd-4dae9edd529e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:34:58.479046 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b290630d-30bc-4bca-bffd-4dae9edd529e None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:34:58.489042 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:34:58.502186 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 565/1138] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Sep 22 02:34:58 2022] POST /identity/v3/users => generated 286 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:35:00.761978 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310dfa58-f422-4aab-9632-62d1f700d798 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:00.762554 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310dfa58-f422-4aab-9632-62d1f700d798 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:00.762554 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310dfa58-f422-4aab-9632-62d1f700d798 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:00.763900 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 574/1139] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:35:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:35:00.769850 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a7080-e337-443f-b503-e542515e431a None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:00.770348 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a7080-e337-443f-b503-e542515e431a None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:00.770458 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a7080-e337-443f-b503-e542515e431a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:00.791131 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-ab4a7080-e337-443f-b503-e542515e431a None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:35:00.792363 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-ab4a7080-e337-443f-b503-e542515e431a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:35:00.857624 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 566/1140] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:35:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:35:01.124310 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310dfa58-f422-4aab-9632-62d1f700d798 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:01.181021 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:01.223959 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310dfa58-f422-4aab-9632-62d1f700d798 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:01.256314 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e531853f-7d44-46b0-9827-fd7fa9855f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:01.256939 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531853f-7d44-46b0-9827-fd7fa9855f84 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:01.257220 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531853f-7d44-46b0-9827-fd7fa9855f84 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:01.257490 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531853f-7d44-46b0-9827-fd7fa9855f84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:01.267182 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e531853f-7d44-46b0-9827-fd7fa9855f84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:35:01.272221 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-e531853f-7d44-46b0-9827-fd7fa9855f84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:35:01.272221 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:35:01.272221 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:35:01.272221 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:35:01.272221 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:35:01.272221 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:35:01.272221 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:35:01.272221 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:35:01.272221 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:35:01.272221 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:35:01.272221 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:35:01.272221 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:35:01.272221 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:35:01.272221 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:35:01.272221 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:35:01.272221 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:35:01.273402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:35:01.273402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:35:01.273402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:35:01.273402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:35:01.273402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:35:01.273402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:35:01.273402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:35:01.273402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:35:01.273402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:35:01.273402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:35:01.273402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:35:01.273402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:35:01.273402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:35:01.273402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:35:01.273402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:35:01.273402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:35:01.274571 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:35:01.274571 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:35:01.274571 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:35:01.274571 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:35:01.274571 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:35:01.274571 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:35:01.274571 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:35:01.274571 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:35:01.274571 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:35:01.274571 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:35:01.274571 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:35:01.274571 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:35:01.274571 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 575/1141] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Sep 22 02:35:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:35:01.279408 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a7080-e337-443f-b503-e542515e431a None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:01.375786 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a7080-e337-443f-b503-e542515e431a None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:01.397753 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:01.410917 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4f6da9-6a30-45d4-98a9-b3cd158d0702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:01.411527 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4f6da9-6a30-45d4-98a9-b3cd158d0702 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:01.411813 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4f6da9-6a30-45d4-98a9-b3cd158d0702 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:01.412091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4f6da9-6a30-45d4-98a9-b3cd158d0702 None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:01.412758 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4f6da9-6a30-45d4-98a9-b3cd158d0702 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:35:01.413064 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4f6da9-6a30-45d4-98a9-b3cd158d0702 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:35:01.414431 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4f6da9-6a30-45d4-98a9-b3cd158d0702 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:35:01.422480 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 567/1142] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:35:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:35:01.428916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e531853f-7d44-46b0-9827-fd7fa9855f84 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:01.525666 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e531853f-7d44-46b0-9827-fd7fa9855f84 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:01.542009 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:01.559267 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae64b8c-e2ef-4f40-be33-1a2038ad3bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:01.559900 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae64b8c-e2ef-4f40-be33-1a2038ad3bcf None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:01.560181 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae64b8c-e2ef-4f40-be33-1a2038ad3bcf None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:01.560454 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae64b8c-e2ef-4f40-be33-1a2038ad3bcf None admin] PATH_INFO: `/v3/users/7da90f58d6e24809a3d38f6a85ab0f48` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:01.573314 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae64b8c-e2ef-4f40-be33-1a2038ad3bcf None admin] RBAC: Authorizing `identity:get_user(user_id=7da90f58d6e24809a3d38f6a85ab0f48)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:35:01.574749 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae64b8c-e2ef-4f40-be33-1a2038ad3bcf None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:35:01.576389 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 576/1143] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:35:01 2022] GET /identity/v3/users/7da90f58d6e24809a3d38f6a85ab0f48 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:35:01.583042 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4f6da9-6a30-45d4-98a9-b3cd158d0702 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:01.682485 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4f6da9-6a30-45d4-98a9-b3cd158d0702 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:01.715665 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227aa9be-73e4-42d3-bb5b-d9aa5823245d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:01.716273 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227aa9be-73e4-42d3-bb5b-d9aa5823245d None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:01.716556 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227aa9be-73e4-42d3-bb5b-d9aa5823245d None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:01.716832 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227aa9be-73e4-42d3-bb5b-d9aa5823245d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:01.768061 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:01.784089 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227aa9be-73e4-42d3-bb5b-d9aa5823245d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:35:01.785479 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227aa9be-73e4-42d3-bb5b-d9aa5823245d None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:35:01.883400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 568/1144] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:35:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 02:35:01.889495 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae64b8c-e2ef-4f40-be33-1a2038ad3bcf None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:01.991971 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae64b8c-e2ef-4f40-be33-1a2038ad3bcf None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:02.027449 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1049387-5b78-4763-8a0e-6048ed51065f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:02.028089 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1049387-5b78-4763-8a0e-6048ed51065f None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:02.028376 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1049387-5b78-4763-8a0e-6048ed51065f None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:02.028656 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1049387-5b78-4763-8a0e-6048ed51065f None admin] PATH_INFO: `/v3/projects/5b9d02733fdc45bca6593b8c17d78c1d` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:02.029784 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1049387-5b78-4763-8a0e-6048ed51065f None admin] RBAC: Authorizing `identity:get_project(project_id=5b9d02733fdc45bca6593b8c17d78c1d)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:35:02.031251 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1049387-5b78-4763-8a0e-6048ed51065f None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:35:02.033575 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 577/1145] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:35:01 2022] GET /identity/v3/projects/5b9d02733fdc45bca6593b8c17d78c1d => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:35:02.039998 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227aa9be-73e4-42d3-bb5b-d9aa5823245d None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:02.125430 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:02.141754 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227aa9be-73e4-42d3-bb5b-d9aa5823245d None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:02.176863 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae2727-c11f-491b-b3ee-9237d6c14cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:02.177472 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae2727-c11f-491b-b3ee-9237d6c14cde None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:02.177757 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae2727-c11f-491b-b3ee-9237d6c14cde None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:02.178036 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae2727-c11f-491b-b3ee-9237d6c14cde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:02.178765 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae2727-c11f-491b-b3ee-9237d6c14cde None admin] RBAC: Adding query filter params (role.id=31a687548f55493b8ea666fe7e350303, scope.project.id=5b9d02733fdc45bca6593b8c17d78c1d, user.id=7da90f58d6e24809a3d38f6a85ab0f48) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:35:02.179111 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae2727-c11f-491b-b3ee-9237d6c14cde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:35:02.180539 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae2727-c11f-491b-b3ee-9237d6c14cde None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:35:02.185921 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 569/1146] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:35:02 2022] GET /identity/v3/role_assignments?user.id=7da90f58d6e24809a3d38f6a85ab0f48&scope.project.id=5b9d02733fdc45bca6593b8c17d78c1d&role.id=31a687548f55493b8ea666fe7e350303 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:35:04.365686 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ad693-299d-40bc-a8a0-4feed086c0eb None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:04.366230 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ad693-299d-40bc-a8a0-4feed086c0eb None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:04.366230 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ad693-299d-40bc-a8a0-4feed086c0eb None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:04.367620 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 578/1147] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:35:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:35:04.372638 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528d7c7-746c-40d1-975f-4e7a6ef27712 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:04.373034 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528d7c7-746c-40d1-975f-4e7a6ef27712 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:04.373190 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528d7c7-746c-40d1-975f-4e7a6ef27712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:04.392178 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-f528d7c7-746c-40d1-975f-4e7a6ef27712 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:35:04.393349 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-f528d7c7-746c-40d1-975f-4e7a6ef27712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:35:04.461238 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 570/1148] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:35:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:35:04.726811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2ad693-299d-40bc-a8a0-4feed086c0eb None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:04.771326 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:04.827945 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2ad693-299d-40bc-a8a0-4feed086c0eb None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:04.861517 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb616ee-f197-4200-8137-78fa0f03fcdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:04.862140 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb616ee-f197-4200-8137-78fa0f03fcdb None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:04.862419 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb616ee-f197-4200-8137-78fa0f03fcdb None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:04.862691 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb616ee-f197-4200-8137-78fa0f03fcdb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:04.872503 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb616ee-f197-4200-8137-78fa0f03fcdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:35:04.877980 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-beb616ee-f197-4200-8137-78fa0f03fcdb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:35:04.877980 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:35:04.877980 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:35:04.877980 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:35:04.877980 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:35:04.877980 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:35:04.877980 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:35:04.877980 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:35:04.877980 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:35:04.877980 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:35:04.877980 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:35:04.877980 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:35:04.877980 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:35:04.877980 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:35:04.877980 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:35:04.877980 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:35:04.879129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:35:04.879129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:35:04.879129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:35:04.879129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:35:04.879129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:35:04.879129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:35:04.879129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:35:04.879129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:35:04.879129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:35:04.879129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:35:04.879129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:35:04.879129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:35:04.879129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:35:04.879129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:35:04.879129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:35:04.879129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:35:04.880206 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:35:04.880206 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:35:04.880206 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:35:04.880206 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:35:04.880206 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:35:04.880206 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:35:04.880206 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:35:04.880206 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:35:04.880206 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:35:04.880206 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:35:04.880206 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:35:04.880206 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:35:04.880206 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 579/1149] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Sep 22 02:35:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:35:04.887451 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528d7c7-746c-40d1-975f-4e7a6ef27712 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:04.963344 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:04.988724 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528d7c7-746c-40d1-975f-4e7a6ef27712 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:05.025100 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e1053-5956-435a-b4a4-3047963aed06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:05.025714 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e1053-5956-435a-b4a4-3047963aed06 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:05.025991 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e1053-5956-435a-b4a4-3047963aed06 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:05.026258 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e1053-5956-435a-b4a4-3047963aed06 None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:05.026950 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3e1053-5956-435a-b4a4-3047963aed06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:35:05.027251 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3e1053-5956-435a-b4a4-3047963aed06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:35:05.028662 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3e1053-5956-435a-b4a4-3047963aed06 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:35:05.037333 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 571/1150] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:35:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:35:05.043460 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb616ee-f197-4200-8137-78fa0f03fcdb None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:05.142344 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb616ee-f197-4200-8137-78fa0f03fcdb None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:05.143906 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:05.178939 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe501b-1dc9-4587-aad0-5fee00185920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:05.179573 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe501b-1dc9-4587-aad0-5fee00185920 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:05.179859 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe501b-1dc9-4587-aad0-5fee00185920 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:05.180135 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe501b-1dc9-4587-aad0-5fee00185920 None admin] PATH_INFO: `/v3/users/7da90f58d6e24809a3d38f6a85ab0f48` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:05.181263 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbe501b-1dc9-4587-aad0-5fee00185920 None admin] RBAC: Authorizing `identity:get_user(user_id=7da90f58d6e24809a3d38f6a85ab0f48)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:35:05.182688 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbe501b-1dc9-4587-aad0-5fee00185920 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:35:05.184305 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 580/1151] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:35:05 2022] GET /identity/v3/users/7da90f58d6e24809a3d38f6a85ab0f48 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:35:05.190350 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e1053-5956-435a-b4a4-3047963aed06 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:05.289564 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e1053-5956-435a-b4a4-3047963aed06 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:05.322046 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb8060-f17b-478f-aa32-acc5c77ca712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:05.322681 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb8060-f17b-478f-aa32-acc5c77ca712 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:05.323001 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb8060-f17b-478f-aa32-acc5c77ca712 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:05.323280 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb8060-f17b-478f-aa32-acc5c77ca712 None admin] PATH_INFO: `/v3/projects/5b9d02733fdc45bca6593b8c17d78c1d` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:05.324429 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb8060-f17b-478f-aa32-acc5c77ca712 None admin] RBAC: Authorizing `identity:get_project(project_id=5b9d02733fdc45bca6593b8c17d78c1d)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:35:05.325872 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb8060-f17b-478f-aa32-acc5c77ca712 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:35:05.327547 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 572/1152] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:35:05 2022] GET /identity/v3/projects/5b9d02733fdc45bca6593b8c17d78c1d => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:35:05.334168 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe501b-1dc9-4587-aad0-5fee00185920 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:05.430516 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe501b-1dc9-4587-aad0-5fee00185920 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:05.463679 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779bd0a-7bdb-4fd0-a632-fa7e626982a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:05.464329 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779bd0a-7bdb-4fd0-a632-fa7e626982a5 None admin] REQUEST_METHOD: `PUT` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:05.464610 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779bd0a-7bdb-4fd0-a632-fa7e626982a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:05.464883 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779bd0a-7bdb-4fd0-a632-fa7e626982a5 None admin] PATH_INFO: `/v3/projects/5b9d02733fdc45bca6593b8c17d78c1d/users/7da90f58d6e24809a3d38f6a85ab0f48/roles/31a687548f55493b8ea666fe7e350303` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:05.466742 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3779bd0a-7bdb-4fd0-a632-fa7e626982a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=5b9d02733fdc45bca6593b8c17d78c1d, user_id=7da90f58d6e24809a3d38f6a85ab0f48, role_id=31a687548f55493b8ea666fe7e350303)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:35:05.468193 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3779bd0a-7bdb-4fd0-a632-fa7e626982a5 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:35:05.482682 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 581/1153] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Sep 22 02:35:05 2022] PUT /identity/v3/projects/5b9d02733fdc45bca6593b8c17d78c1d/users/7da90f58d6e24809a3d38f6a85ab0f48/roles/31a687548f55493b8ea666fe7e350303 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 02:35:07.653902 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fcc90-aa96-4a10-aed0-661cdb115782 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:07.654477 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fcc90-aa96-4a10-aed0-661cdb115782 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:07.654477 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fcc90-aa96-4a10-aed0-661cdb115782 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:07.655837 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 573/1154] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:35:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:35:07.660990 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a13f4-314e-4477-9e1f-90d8b9f2bac6 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:07.661411 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a13f4-314e-4477-9e1f-90d8b9f2bac6 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:07.661522 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a13f4-314e-4477-9e1f-90d8b9f2bac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:07.679957 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-bc8a13f4-314e-4477-9e1f-90d8b9f2bac6 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:35:07.681199 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-bc8a13f4-314e-4477-9e1f-90d8b9f2bac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:35:07.704783 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:07.747365 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 582/1155] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:35:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:35:08.017554 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fcc90-aa96-4a10-aed0-661cdb115782 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:08.029260 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:08.116848 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fcc90-aa96-4a10-aed0-661cdb115782 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:08.151094 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479f173-dbc6-4d8d-b465-d7e2ee235cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:08.151721 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479f173-dbc6-4d8d-b465-d7e2ee235cec None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:08.152007 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479f173-dbc6-4d8d-b465-d7e2ee235cec None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:08.152282 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479f173-dbc6-4d8d-b465-d7e2ee235cec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:08.162055 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6479f173-dbc6-4d8d-b465-d7e2ee235cec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:35:08.167091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-6479f173-dbc6-4d8d-b465-d7e2ee235cec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:35:08.167091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:35:08.167091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:35:08.167091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:35:08.167091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:35:08.167091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:35:08.167091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:35:08.167091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:35:08.167091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:35:08.167091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:35:08.167091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:35:08.167091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:35:08.167091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:35:08.167091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:35:08.167091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:35:08.167091 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:35:08.168226 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:35:08.168226 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:35:08.168226 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:35:08.168226 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:35:08.168226 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:35:08.168226 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:35:08.168226 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:35:08.168226 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:35:08.168226 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:35:08.168226 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:35:08.168226 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:35:08.168226 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:35:08.168226 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:35:08.168226 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:35:08.168226 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:35:08.168226 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:35:08.169259 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:35:08.169259 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:35:08.169259 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:35:08.169259 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:35:08.169259 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:35:08.169259 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:35:08.169259 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:35:08.169259 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:35:08.169259 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:35:08.169259 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:35:08.169259 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:35:08.169259 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:35:08.169259 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 574/1156] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Sep 22 02:35:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:35:08.174313 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a13f4-314e-4477-9e1f-90d8b9f2bac6 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:08.272794 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a13f4-314e-4477-9e1f-90d8b9f2bac6 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:08.308036 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef64bf-c80e-4307-850a-61e5bb355964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:08.308656 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef64bf-c80e-4307-850a-61e5bb355964 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:08.308943 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef64bf-c80e-4307-850a-61e5bb355964 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:08.309293 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef64bf-c80e-4307-850a-61e5bb355964 None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:08.310300 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ef64bf-c80e-4307-850a-61e5bb355964 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:35:08.310613 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ef64bf-c80e-4307-850a-61e5bb355964 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:35:08.313667 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ef64bf-c80e-4307-850a-61e5bb355964 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:35:08.324270 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 583/1157] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:35:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:35:08.331651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479f173-dbc6-4d8d-b465-d7e2ee235cec None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:08.397792 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:08.431228 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479f173-dbc6-4d8d-b465-d7e2ee235cec None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:08.464820 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57635e2a-fb09-4968-af6b-c0566c559664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:08.465450 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57635e2a-fb09-4968-af6b-c0566c559664 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:08.465729 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57635e2a-fb09-4968-af6b-c0566c559664 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:08.466000 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57635e2a-fb09-4968-af6b-c0566c559664 None admin] PATH_INFO: `/v3/users/7da90f58d6e24809a3d38f6a85ab0f48` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:08.467134 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57635e2a-fb09-4968-af6b-c0566c559664 None admin] RBAC: Authorizing `identity:get_user(user_id=7da90f58d6e24809a3d38f6a85ab0f48)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:35:08.468565 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57635e2a-fb09-4968-af6b-c0566c559664 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:35:08.470781 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 575/1158] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:35:08 2022] GET /identity/v3/users/7da90f58d6e24809a3d38f6a85ab0f48 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:35:08.476665 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef64bf-c80e-4307-850a-61e5bb355964 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:08.501086 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:08.575231 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef64bf-c80e-4307-850a-61e5bb355964 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:08.609171 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1313bc4d-3951-4b75-85d2-c3a8133462b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:08.609816 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313bc4d-3951-4b75-85d2-c3a8133462b7 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:08.610102 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313bc4d-3951-4b75-85d2-c3a8133462b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:08.610383 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313bc4d-3951-4b75-85d2-c3a8133462b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:08.676161 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1313bc4d-3951-4b75-85d2-c3a8133462b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:35:08.677607 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1313bc4d-3951-4b75-85d2-c3a8133462b7 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:35:08.703111 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:08.780470 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 584/1159] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:35:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 02:35:08.786452 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57635e2a-fb09-4968-af6b-c0566c559664 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:08.885143 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57635e2a-fb09-4968-af6b-c0566c559664 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:08.920231 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07bd77-859b-45e0-96af-3dcbcd39bd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:08.920860 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07bd77-859b-45e0-96af-3dcbcd39bd99 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:08.921143 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07bd77-859b-45e0-96af-3dcbcd39bd99 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:08.921416 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07bd77-859b-45e0-96af-3dcbcd39bd99 None admin] PATH_INFO: `/v3/projects/5b9d02733fdc45bca6593b8c17d78c1d` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:08.922528 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e07bd77-859b-45e0-96af-3dcbcd39bd99 None admin] RBAC: Authorizing `identity:get_project(project_id=5b9d02733fdc45bca6593b8c17d78c1d)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:35:08.923980 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e07bd77-859b-45e0-96af-3dcbcd39bd99 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:35:08.925642 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 576/1160] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:35:08 2022] GET /identity/v3/projects/5b9d02733fdc45bca6593b8c17d78c1d => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:35:08.931847 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1313bc4d-3951-4b75-85d2-c3a8133462b7 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:09.028599 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1313bc4d-3951-4b75-85d2-c3a8133462b7 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:09.051255 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:09.063665 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f7a7f-a917-4c36-9062-ed321eec3489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:09.064270 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f7a7f-a917-4c36-9062-ed321eec3489 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:09.064551 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f7a7f-a917-4c36-9062-ed321eec3489 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:09.064824 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f7a7f-a917-4c36-9062-ed321eec3489 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:09.065545 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f7a7f-a917-4c36-9062-ed321eec3489 None admin] RBAC: Adding query filter params (role.id=31a687548f55493b8ea666fe7e350303, scope.project.id=5b9d02733fdc45bca6593b8c17d78c1d, user.id=7da90f58d6e24809a3d38f6a85ab0f48) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:35:09.065854 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f7a7f-a917-4c36-9062-ed321eec3489 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:35:09.067314 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f7a7f-a917-4c36-9062-ed321eec3489 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:35:09.073079 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 585/1161] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:35:08 2022] GET /identity/v3/role_assignments?user.id=7da90f58d6e24809a3d38f6a85ab0f48&scope.project.id=5b9d02733fdc45bca6593b8c17d78c1d&role.id=31a687548f55493b8ea666fe7e350303 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:35:11.228844 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f8680f-a3e3-44fb-9bdb-f91453ee5361 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:11.229374 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f8680f-a3e3-44fb-9bdb-f91453ee5361 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:11.229374 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f8680f-a3e3-44fb-9bdb-f91453ee5361 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:11.230702 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 577/1162] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:35:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:35:11.236432 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a03d9-56d1-4dd9-89d7-d72a4ffec4a4 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:11.236812 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a03d9-56d1-4dd9-89d7-d72a4ffec4a4 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:11.236984 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a03d9-56d1-4dd9-89d7-d72a4ffec4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:11.255097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-0f7a03d9-56d1-4dd9-89d7-d72a4ffec4a4 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:35:11.256304 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-0f7a03d9-56d1-4dd9-89d7-d72a4ffec4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:35:11.325426 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 586/1163] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:35:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:35:11.596962 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f8680f-a3e3-44fb-9bdb-f91453ee5361 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:11.600803 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:11.697655 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f8680f-a3e3-44fb-9bdb-f91453ee5361 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:11.730618 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7576c9-fa8e-40bf-86ba-1595c09fb5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:11.731251 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7576c9-fa8e-40bf-86ba-1595c09fb5cc None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:11.731536 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7576c9-fa8e-40bf-86ba-1595c09fb5cc None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:11.731811 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7576c9-fa8e-40bf-86ba-1595c09fb5cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:11.796776 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7576c9-fa8e-40bf-86ba-1595c09fb5cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:35:11.798436 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7576c9-fa8e-40bf-86ba-1595c09fb5cc None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:35:11.801073 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:11.893939 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 578/1164] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:35:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 02:35:11.900096 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a03d9-56d1-4dd9-89d7-d72a4ffec4a4 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:11.997613 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a03d9-56d1-4dd9-89d7-d72a4ffec4a4 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:12.016764 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:12.036113 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43c234-1cb4-431a-8c17-92b29894bfe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:12.036747 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43c234-1cb4-431a-8c17-92b29894bfe7 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:12.037027 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43c234-1cb4-431a-8c17-92b29894bfe7 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:12.037301 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43c234-1cb4-431a-8c17-92b29894bfe7 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:12.044153 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a43c234-1cb4-431a-8c17-92b29894bfe7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:35:12.045595 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a43c234-1cb4-431a-8c17-92b29894bfe7 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:35:12.057524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-7a43c234-1cb4-431a-8c17-92b29894bfe7 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 22 02:35:12.057524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:35:12.057524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 22 02:35:12.057524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 22 02:35:12.057524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:35:12.057524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:35:12.057524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:35:12.057524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:35:12.057524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 22 02:35:12.057524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:35:12.057524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 02:35:12.057524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:35:12.057524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:35:12.057524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:35:12.057524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:35:12.057524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:35:12.057524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:35:12.058877 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:35:12.058877 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:35:12.058877 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:35:12.058877 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:35:12.058877 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:35:12.058877 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:35:12.058877 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:35:12.058877 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:35:12.058877 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:35:12.058877 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:35:12.058877 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:35:12.058877 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:35:12.058877 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:35:12.058877 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:35:12.058877 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:35:12.058877 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:35:12.060020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:35:12.060020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:35:12.060020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:35:12.060020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:35:12.060020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:35:12.060020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:35:12.060020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:35:12.060020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:35:12.060020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:35:12.060020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:35:12.060020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 22 02:35:12.060020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:35:12.060020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 22 02:35:12.060020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:35:12.060020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 587/1165] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Sep 22 02:35:11 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:35:12.065353 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7576c9-fa8e-40bf-86ba-1595c09fb5cc None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:12.169316 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7576c9-fa8e-40bf-86ba-1595c09fb5cc None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:12.171645 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:12.204450 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7024ca-1f1e-42a2-b6c5-3ba5aa3f81e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:12.205053 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7024ca-1f1e-42a2-b6c5-3ba5aa3f81e7 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:12.205342 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7024ca-1f1e-42a2-b6c5-3ba5aa3f81e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:12.205619 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7024ca-1f1e-42a2-b6c5-3ba5aa3f81e7 None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:12.206268 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7024ca-1f1e-42a2-b6c5-3ba5aa3f81e7 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:35:12.206577 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7024ca-1f1e-42a2-b6c5-3ba5aa3f81e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:35:12.207988 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7024ca-1f1e-42a2-b6c5-3ba5aa3f81e7 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:35:12.215242 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 579/1166] 10.222.0.20 () {56 vars in 1115 bytes} [Thu Sep 22 02:35:12 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:35:14.372377 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3f9e1-e84c-4d1e-8bbe-b2c151d941ae None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:14.372943 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3f9e1-e84c-4d1e-8bbe-b2c151d941ae None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:14.372943 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3f9e1-e84c-4d1e-8bbe-b2c151d941ae None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:14.374252 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 588/1167] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:35:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:35:14.379639 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350a653-df90-41d9-8dc9-2082ac41dfcd None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:14.380020 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350a653-df90-41d9-8dc9-2082ac41dfcd None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:14.380201 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350a653-df90-41d9-8dc9-2082ac41dfcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:14.398938 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-9350a653-df90-41d9-8dc9-2082ac41dfcd None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:35:14.400137 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-9350a653-df90-41d9-8dc9-2082ac41dfcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:35:14.467536 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 580/1168] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:35:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:35:14.733083 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3f9e1-e84c-4d1e-8bbe-b2c151d941ae None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:14.832465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3f9e1-e84c-4d1e-8bbe-b2c151d941ae None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:14.865242 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd2746-eae4-4e08-87b3-4829fd0979bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:14.865845 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd2746-eae4-4e08-87b3-4829fd0979bb None admin] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:14.866123 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd2746-eae4-4e08-87b3-4829fd0979bb None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:14.866390 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd2746-eae4-4e08-87b3-4829fd0979bb None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:14.867086 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bd2746-eae4-4e08-87b3-4829fd0979bb None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:35:14.868448 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bd2746-eae4-4e08-87b3-4829fd0979bb None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:35:14.884313 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 589/1169] 10.222.0.20 () {60 vars in 1138 bytes} [Thu Sep 22 02:35:14 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:35:17.074528 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8233b2-b42c-438b-af03-2e8cc645aee8 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:17.075107 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8233b2-b42c-438b-af03-2e8cc645aee8 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:17.075107 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8233b2-b42c-438b-af03-2e8cc645aee8 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:17.076440 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 581/1170] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:35:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:35:17.083084 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275c2b9-4e94-4172-af38-1dd85b6e6d94 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:17.083525 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275c2b9-4e94-4172-af38-1dd85b6e6d94 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:17.083649 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275c2b9-4e94-4172-af38-1dd85b6e6d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:17.102192 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-4275c2b9-4e94-4172-af38-1dd85b6e6d94 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:35:17.103446 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-4275c2b9-4e94-4172-af38-1dd85b6e6d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:35:17.132397 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:17.172685 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 590/1171] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:35:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:35:17.451173 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8233b2-b42c-438b-af03-2e8cc645aee8 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:17.544369 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:17.550141 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8233b2-b42c-438b-af03-2e8cc645aee8 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:17.583751 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab053f1b-cb4d-4608-ba6c-681cd55427ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:17.584373 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab053f1b-cb4d-4608-ba6c-681cd55427ae None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:17.584653 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab053f1b-cb4d-4608-ba6c-681cd55427ae None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:17.584924 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab053f1b-cb4d-4608-ba6c-681cd55427ae None admin] PATH_INFO: `/v3/domains/e9d9df7df48d4abbb5d9a07073c274f6` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:17.594946 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab053f1b-cb4d-4608-ba6c-681cd55427ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=e9d9df7df48d4abbb5d9a07073c274f6)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:35:17.596373 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab053f1b-cb4d-4608-ba6c-681cd55427ae None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:35:17.597987 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 582/1172] 10.222.0.20 () {56 vars in 1183 bytes} [Thu Sep 22 02:35:17 2022] GET /identity/v3/domains/e9d9df7df48d4abbb5d9a07073c274f6 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:35:17.603936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275c2b9-4e94-4172-af38-1dd85b6e6d94 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:17.708780 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275c2b9-4e94-4172-af38-1dd85b6e6d94 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:17.753096 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88f7d8-9f2b-42f7-a623-1181421de664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:17.753714 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88f7d8-9f2b-42f7-a623-1181421de664 None admin] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:17.753992 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88f7d8-9f2b-42f7-a623-1181421de664 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:17.754270 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88f7d8-9f2b-42f7-a623-1181421de664 None admin] PATH_INFO: `/v3/projects` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:17.754950 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88f7d8-9f2b-42f7-a623-1181421de664 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:35:17.756394 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88f7d8-9f2b-42f7-a623-1181421de664 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:35:17.784635 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 591/1173] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Sep 22 02:35:17 2022] POST /identity/v3/projects => generated 361 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:35:19.989485 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e67e51-b217-4246-ac3a-67bc3b568237 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:19.990024 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e67e51-b217-4246-ac3a-67bc3b568237 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:19.990024 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e67e51-b217-4246-ac3a-67bc3b568237 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:19.991433 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 583/1174] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:35:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:35:19.996889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ceffb-c5b5-41b8-8513-4de8ae4aa388 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:19.997329 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ceffb-c5b5-41b8-8513-4de8ae4aa388 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:19.997425 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ceffb-c5b5-41b8-8513-4de8ae4aa388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:20.015663 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-ff3ceffb-c5b5-41b8-8513-4de8ae4aa388 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:35:20.016854 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-ff3ceffb-c5b5-41b8-8513-4de8ae4aa388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:35:20.031288 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:20.086501 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 592/1175] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:35:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:35:20.368611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e67e51-b217-4246-ac3a-67bc3b568237 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:20.473362 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e67e51-b217-4246-ac3a-67bc3b568237 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:20.510347 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a580dfb-54a0-45b1-b31c-a271baed5db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:20.511017 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a580dfb-54a0-45b1-b31c-a271baed5db7 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:20.511322 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a580dfb-54a0-45b1-b31c-a271baed5db7 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:20.511580 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a580dfb-54a0-45b1-b31c-a271baed5db7 None admin] PATH_INFO: `/v3/domains/e9d9df7df48d4abbb5d9a07073c274f6` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:20.512701 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a580dfb-54a0-45b1-b31c-a271baed5db7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=e9d9df7df48d4abbb5d9a07073c274f6)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:35:20.514169 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a580dfb-54a0-45b1-b31c-a271baed5db7 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:35:20.515829 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 584/1176] 10.222.0.20 () {56 vars in 1183 bytes} [Thu Sep 22 02:35:20 2022] GET /identity/v3/domains/e9d9df7df48d4abbb5d9a07073c274f6 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:35:20.521884 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ceffb-c5b5-41b8-8513-4de8ae4aa388 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:20.619371 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ceffb-c5b5-41b8-8513-4de8ae4aa388 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:20.652978 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba5338-e17a-4521-8a9f-a43a221b41f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:20.653619 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba5338-e17a-4521-8a9f-a43a221b41f7 None admin] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:20.653905 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba5338-e17a-4521-8a9f-a43a221b41f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:20.654184 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba5338-e17a-4521-8a9f-a43a221b41f7 None admin] PATH_INFO: `/v3/users` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:20.654859 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daba5338-e17a-4521-8a9f-a43a221b41f7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:35:20.656851 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daba5338-e17a-4521-8a9f-a43a221b41f7 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:35:20.678940 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 593/1177] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Sep 22 02:35:20 2022] POST /identity/v3/users => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 02:35:22.883727 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4b989-aa45-4dd3-868a-64e72ddacf5a None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:22.884266 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4b989-aa45-4dd3-868a-64e72ddacf5a None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:22.884266 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4b989-aa45-4dd3-868a-64e72ddacf5a None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:22.885625 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 585/1178] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:35:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:35:22.890961 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e90c3-a646-461f-b2b5-9473d181e6b3 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:22.891344 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e90c3-a646-461f-b2b5-9473d181e6b3 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:22.891509 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e90c3-a646-461f-b2b5-9473d181e6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:22.898950 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:22.910869 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-425e90c3-a646-461f-b2b5-9473d181e6b3 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:35:22.912031 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-425e90c3-a646-461f-b2b5-9473d181e6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:35:22.976302 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 594/1179] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:35:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:35:23.241427 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b4b989-aa45-4dd3-868a-64e72ddacf5a None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:23.268228 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:23.340569 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b4b989-aa45-4dd3-868a-64e72ddacf5a None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:23.373781 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d676f-f5f5-4976-9f49-d448c7c72466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:23.375038 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d676f-f5f5-4976-9f49-d448c7c72466 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:23.375329 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d676f-f5f5-4976-9f49-d448c7c72466 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:23.375605 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d676f-f5f5-4976-9f49-d448c7c72466 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:23.385417 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d676f-f5f5-4976-9f49-d448c7c72466 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:35:23.390266 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-915d676f-f5f5-4976-9f49-d448c7c72466 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:35:23.390266 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:35:23.390266 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:35:23.390266 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:35:23.390266 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:35:23.390266 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:35:23.390266 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:35:23.390266 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:35:23.390266 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:35:23.390266 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:35:23.390266 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:35:23.390266 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:35:23.390266 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:35:23.390266 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:35:23.390266 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:35:23.390266 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:35:23.391325 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:35:23.391325 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:35:23.391325 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:35:23.391325 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:35:23.391325 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:35:23.391325 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:35:23.391325 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:35:23.391325 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:35:23.391325 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:35:23.391325 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:35:23.391325 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:35:23.391325 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:35:23.391325 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:35:23.391325 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:35:23.391325 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:35:23.391325 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:35:23.392220 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:35:23.392220 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:35:23.392220 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:35:23.392220 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:35:23.392220 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:35:23.392220 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:35:23.392220 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:35:23.392220 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:35:23.392220 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:35:23.392220 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:35:23.392220 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:35:23.392220 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:35:23.392220 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 586/1180] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Sep 22 02:35:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:35:23.397889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e90c3-a646-461f-b2b5-9473d181e6b3 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:23.495538 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e90c3-a646-461f-b2b5-9473d181e6b3 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:23.529687 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad66ee-a445-4d02-aae1-27f45b5e5c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:23.530299 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad66ee-a445-4d02-aae1-27f45b5e5c9f None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:23.530585 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad66ee-a445-4d02-aae1-27f45b5e5c9f None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:23.530887 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad66ee-a445-4d02-aae1-27f45b5e5c9f None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:23.531561 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad66ee-a445-4d02-aae1-27f45b5e5c9f None admin] RBAC: Adding query filter params (name=admin) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:35:23.531870 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad66ee-a445-4d02-aae1-27f45b5e5c9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:35:23.533245 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad66ee-a445-4d02-aae1-27f45b5e5c9f None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:35:23.534603 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:23.542103 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 595/1181] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:35:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:35:23.548937 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d676f-f5f5-4976-9f49-d448c7c72466 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:23.633105 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:23.648237 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d676f-f5f5-4976-9f49-d448c7c72466 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:23.685093 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fc3643-023a-45a3-b5a6-bc103a0811ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:23.685727 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc3643-023a-45a3-b5a6-bc103a0811ca None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:23.686008 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc3643-023a-45a3-b5a6-bc103a0811ca None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:23.686285 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc3643-023a-45a3-b5a6-bc103a0811ca None admin] PATH_INFO: `/v3/users/784c7ded33d24463a0d870914fc6d339` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:23.699438 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fc3643-023a-45a3-b5a6-bc103a0811ca None admin] RBAC: Authorizing `identity:get_user(user_id=784c7ded33d24463a0d870914fc6d339)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:35:23.700892 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fc3643-023a-45a3-b5a6-bc103a0811ca None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:35:23.702554 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 587/1182] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:35:23 2022] GET /identity/v3/users/784c7ded33d24463a0d870914fc6d339 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:35:23.708408 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad66ee-a445-4d02-aae1-27f45b5e5c9f None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:23.807662 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad66ee-a445-4d02-aae1-27f45b5e5c9f None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:23.839645 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaf3e34-40c4-4e67-88b6-624e5e4c3d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:23.840236 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf3e34-40c4-4e67-88b6-624e5e4c3d03 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:23.840515 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf3e34-40c4-4e67-88b6-624e5e4c3d03 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:23.840785 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf3e34-40c4-4e67-88b6-624e5e4c3d03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:23.900516 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:23.907773 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaf3e34-40c4-4e67-88b6-624e5e4c3d03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:35:23.909241 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaf3e34-40c4-4e67-88b6-624e5e4c3d03 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:35:24.004945 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 596/1183] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:35:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 02:35:24.011086 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fc3643-023a-45a3-b5a6-bc103a0811ca None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:24.114759 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fc3643-023a-45a3-b5a6-bc103a0811ca None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:24.149953 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55549193-0c0a-477e-8049-be6e49f503f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:24.150591 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55549193-0c0a-477e-8049-be6e49f503f6 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:24.150904 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55549193-0c0a-477e-8049-be6e49f503f6 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:24.151186 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55549193-0c0a-477e-8049-be6e49f503f6 None admin] PATH_INFO: `/v3/projects/5269838069d64b3e9d74d266d13a54be` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:24.152317 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55549193-0c0a-477e-8049-be6e49f503f6 None admin] RBAC: Authorizing `identity:get_project(project_id=5269838069d64b3e9d74d266d13a54be)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:35:24.153772 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55549193-0c0a-477e-8049-be6e49f503f6 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:35:24.155502 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 588/1184] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:35:24 2022] GET /identity/v3/projects/5269838069d64b3e9d74d266d13a54be => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:35:24.162741 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaf3e34-40c4-4e67-88b6-624e5e4c3d03 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:24.254751 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:24.261195 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaf3e34-40c4-4e67-88b6-624e5e4c3d03 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:24.295494 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964d63d-48c9-4351-a64c-ca86ef87e07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:24.296090 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964d63d-48c9-4351-a64c-ca86ef87e07f None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:24.296879 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964d63d-48c9-4351-a64c-ca86ef87e07f None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:24.297146 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964d63d-48c9-4351-a64c-ca86ef87e07f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:24.297859 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0964d63d-48c9-4351-a64c-ca86ef87e07f None admin] RBAC: Adding query filter params (role.id=31a687548f55493b8ea666fe7e350303, scope.project.id=5269838069d64b3e9d74d266d13a54be, user.id=784c7ded33d24463a0d870914fc6d339) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:35:24.298164 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0964d63d-48c9-4351-a64c-ca86ef87e07f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:35:24.299596 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0964d63d-48c9-4351-a64c-ca86ef87e07f None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:35:24.304818 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 597/1185] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:35:24 2022] GET /identity/v3/role_assignments?user.id=784c7ded33d24463a0d870914fc6d339&scope.project.id=5269838069d64b3e9d74d266d13a54be&role.id=31a687548f55493b8ea666fe7e350303 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:35:26.442532 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c759f9-8a13-4b50-831e-eca70ad02243 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:26.443126 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c759f9-8a13-4b50-831e-eca70ad02243 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:26.443126 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c759f9-8a13-4b50-831e-eca70ad02243 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:26.444444 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 589/1186] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:35:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:35:26.449973 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b88fd-9159-464f-8620-ee5d9e1c3d81 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:26.450393 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b88fd-9159-464f-8620-ee5d9e1c3d81 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:26.450524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b88fd-9159-464f-8620-ee5d9e1c3d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:26.469202 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-113b88fd-9159-464f-8620-ee5d9e1c3d81 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:35:26.470416 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-113b88fd-9159-464f-8620-ee5d9e1c3d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:35:26.538201 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 598/1187] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:35:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:35:26.807980 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c759f9-8a13-4b50-831e-eca70ad02243 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:26.823470 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:26.911291 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c759f9-8a13-4b50-831e-eca70ad02243 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:26.944804 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89ba2fb-8b15-4925-89c7-d14209ca2297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:26.945424 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ba2fb-8b15-4925-89c7-d14209ca2297 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:26.945703 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ba2fb-8b15-4925-89c7-d14209ca2297 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:26.945978 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ba2fb-8b15-4925-89c7-d14209ca2297 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:26.955795 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89ba2fb-8b15-4925-89c7-d14209ca2297 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:35:26.960553 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-c89ba2fb-8b15-4925-89c7-d14209ca2297 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:35:26.960553 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:35:26.960553 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:35:26.960553 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:35:26.960553 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:35:26.960553 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:35:26.960553 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:35:26.960553 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:35:26.960553 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:35:26.960553 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:35:26.960553 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:35:26.960553 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:35:26.960553 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:35:26.960553 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:35:26.960553 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:35:26.960553 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:35:26.961682 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:35:26.961682 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:35:26.961682 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:35:26.961682 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:35:26.961682 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:35:26.961682 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:35:26.961682 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:35:26.961682 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:35:26.961682 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:35:26.961682 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:35:26.961682 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:35:26.961682 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:35:26.961682 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:35:26.961682 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:35:26.961682 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:35:26.961682 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:35:26.962784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:35:26.962784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:35:26.962784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:35:26.962784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:35:26.962784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:35:26.962784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:35:26.962784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:35:26.962784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:35:26.962784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:35:26.962784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:35:26.962784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:35:26.962784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:35:26.962784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 590/1188] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Sep 22 02:35:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:35:26.968619 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113b88fd-9159-464f-8620-ee5d9e1c3d81 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:27.060030 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:27.069849 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113b88fd-9159-464f-8620-ee5d9e1c3d81 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:27.104064 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f9132f-8a4b-4711-81ca-6584d813fd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:27.104686 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9132f-8a4b-4711-81ca-6584d813fd68 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:27.104970 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9132f-8a4b-4711-81ca-6584d813fd68 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:27.105246 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9132f-8a4b-4711-81ca-6584d813fd68 None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:27.105916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f9132f-8a4b-4711-81ca-6584d813fd68 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:35:27.106226 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f9132f-8a4b-4711-81ca-6584d813fd68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:35:27.107631 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f9132f-8a4b-4711-81ca-6584d813fd68 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:35:27.116297 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 599/1189] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:35:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:35:27.122181 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89ba2fb-8b15-4925-89c7-d14209ca2297 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:27.194729 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:27.224557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89ba2fb-8b15-4925-89c7-d14209ca2297 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:27.258972 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d730e2-5dfb-45ae-929f-1bef27dcda0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:27.259601 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d730e2-5dfb-45ae-929f-1bef27dcda0b None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:27.259880 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d730e2-5dfb-45ae-929f-1bef27dcda0b None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:27.260150 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d730e2-5dfb-45ae-929f-1bef27dcda0b None admin] PATH_INFO: `/v3/users/784c7ded33d24463a0d870914fc6d339` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:27.261263 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d730e2-5dfb-45ae-929f-1bef27dcda0b None admin] RBAC: Authorizing `identity:get_user(user_id=784c7ded33d24463a0d870914fc6d339)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:35:27.262690 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d730e2-5dfb-45ae-929f-1bef27dcda0b None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:35:27.264296 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 591/1190] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:35:27 2022] GET /identity/v3/users/784c7ded33d24463a0d870914fc6d339 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:35:27.270176 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f9132f-8a4b-4711-81ca-6584d813fd68 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:27.368393 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f9132f-8a4b-4711-81ca-6584d813fd68 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:27.401825 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cbada0-4df5-4494-adfa-cffd57925068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:27.402459 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbada0-4df5-4494-adfa-cffd57925068 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:27.402739 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbada0-4df5-4494-adfa-cffd57925068 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:27.403039 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbada0-4df5-4494-adfa-cffd57925068 None admin] PATH_INFO: `/v3/projects/5269838069d64b3e9d74d266d13a54be` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:27.404150 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cbada0-4df5-4494-adfa-cffd57925068 None admin] RBAC: Authorizing `identity:get_project(project_id=5269838069d64b3e9d74d266d13a54be)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:35:27.406152 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cbada0-4df5-4494-adfa-cffd57925068 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:35:27.407820 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 600/1191] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:35:27 2022] GET /identity/v3/projects/5269838069d64b3e9d74d266d13a54be => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:35:27.413865 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d730e2-5dfb-45ae-929f-1bef27dcda0b None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:27.516830 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d730e2-5dfb-45ae-929f-1bef27dcda0b None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:27.553317 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7050d61-f632-4328-97f1-6e4c97d58075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:27.553985 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7050d61-f632-4328-97f1-6e4c97d58075 None admin] REQUEST_METHOD: `PUT` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:27.554269 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7050d61-f632-4328-97f1-6e4c97d58075 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:27.554542 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7050d61-f632-4328-97f1-6e4c97d58075 None admin] PATH_INFO: `/v3/projects/5269838069d64b3e9d74d266d13a54be/users/784c7ded33d24463a0d870914fc6d339/roles/31a687548f55493b8ea666fe7e350303` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:27.556481 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7050d61-f632-4328-97f1-6e4c97d58075 None admin] RBAC: Authorizing `identity:create_grant(project_id=5269838069d64b3e9d74d266d13a54be, user_id=784c7ded33d24463a0d870914fc6d339, role_id=31a687548f55493b8ea666fe7e350303)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:35:27.557916 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7050d61-f632-4328-97f1-6e4c97d58075 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:35:27.567355 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:27.573135 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 592/1192] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Sep 22 02:35:27 2022] PUT /identity/v3/projects/5269838069d64b3e9d74d266d13a54be/users/784c7ded33d24463a0d870914fc6d339/roles/31a687548f55493b8ea666fe7e350303 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 02:35:29.756542 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc1ec4-553c-4a25-9bf0-8837d6b38c47 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:29.757093 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc1ec4-553c-4a25-9bf0-8837d6b38c47 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:29.757093 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc1ec4-553c-4a25-9bf0-8837d6b38c47 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:29.758448 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 601/1193] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:35:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:35:29.763902 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273a503-3b2d-410e-a701-645b8c95ae8e None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:29.764312 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273a503-3b2d-410e-a701-645b8c95ae8e None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:29.764451 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273a503-3b2d-410e-a701-645b8c95ae8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:29.798158 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-0273a503-3b2d-410e-a701-645b8c95ae8e None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:35:29.801211 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-0273a503-3b2d-410e-a701-645b8c95ae8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:35:29.874177 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 593/1194] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:35:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:35:30.154111 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bc1ec4-553c-4a25-9bf0-8837d6b38c47 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:30.179525 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:30.267361 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bc1ec4-553c-4a25-9bf0-8837d6b38c47 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:30.302227 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc2c2f4-c05f-41c9-a3d9-d5d4366346fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:30.302769 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc2c2f4-c05f-41c9-a3d9-d5d4366346fe None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:30.303081 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc2c2f4-c05f-41c9-a3d9-d5d4366346fe None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:30.303349 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc2c2f4-c05f-41c9-a3d9-d5d4366346fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:30.314025 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc2c2f4-c05f-41c9-a3d9-d5d4366346fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:35:30.318894 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-0cc2c2f4-c05f-41c9-a3d9-d5d4366346fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:35:30.318894 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:35:30.318894 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:35:30.318894 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:35:30.318894 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:35:30.318894 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:35:30.318894 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:35:30.318894 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:35:30.318894 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:35:30.318894 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:35:30.318894 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:35:30.318894 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:35:30.318894 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:35:30.318894 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:35:30.318894 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:35:30.318894 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:35:30.320132 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:35:30.320132 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:35:30.320132 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:35:30.320132 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:35:30.320132 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:35:30.320132 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:35:30.320132 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:35:30.320132 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:35:30.320132 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:35:30.320132 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:35:30.320132 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:35:30.320132 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:35:30.320132 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:35:30.320132 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:35:30.320132 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:35:30.320132 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:35:30.321390 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:35:30.321390 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:35:30.321390 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:35:30.321390 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:35:30.321390 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:35:30.321390 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:35:30.321390 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:35:30.321390 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:35:30.321390 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:35:30.321390 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:35:30.321390 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:35:30.321390 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:35:30.321390 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 602/1195] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Sep 22 02:35:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:35:30.326478 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0273a503-3b2d-410e-a701-645b8c95ae8e None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:30.435679 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0273a503-3b2d-410e-a701-645b8c95ae8e None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:30.478007 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:30.485618 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049afb6f-b40c-4f72-9208-9f31f2ac7974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:30.486244 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049afb6f-b40c-4f72-9208-9f31f2ac7974 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:30.486530 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049afb6f-b40c-4f72-9208-9f31f2ac7974 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:30.486811 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049afb6f-b40c-4f72-9208-9f31f2ac7974 None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:30.487515 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049afb6f-b40c-4f72-9208-9f31f2ac7974 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:35:30.487820 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049afb6f-b40c-4f72-9208-9f31f2ac7974 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:35:30.489445 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049afb6f-b40c-4f72-9208-9f31f2ac7974 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:35:30.500599 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 594/1196] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:35:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:35:30.507053 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc2c2f4-c05f-41c9-a3d9-d5d4366346fe None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:30.593393 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:30.618185 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc2c2f4-c05f-41c9-a3d9-d5d4366346fe None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:30.652668 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0e6dde-c3fb-40d2-8664-1465a97749e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:30.653310 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e6dde-c3fb-40d2-8664-1465a97749e9 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:30.653643 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e6dde-c3fb-40d2-8664-1465a97749e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:30.653972 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e6dde-c3fb-40d2-8664-1465a97749e9 None admin] PATH_INFO: `/v3/users/784c7ded33d24463a0d870914fc6d339` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:30.655278 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0e6dde-c3fb-40d2-8664-1465a97749e9 None admin] RBAC: Authorizing `identity:get_user(user_id=784c7ded33d24463a0d870914fc6d339)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:35:30.656707 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0e6dde-c3fb-40d2-8664-1465a97749e9 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:35:30.658294 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 603/1197] 10.222.0.20 () {56 vars in 1177 bytes} [Thu Sep 22 02:35:30 2022] GET /identity/v3/users/784c7ded33d24463a0d870914fc6d339 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:35:30.664174 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049afb6f-b40c-4f72-9208-9f31f2ac7974 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:30.767946 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049afb6f-b40c-4f72-9208-9f31f2ac7974 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:30.809504 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f174769-7531-42b1-93ea-48ca7c06f5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:30.810155 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f174769-7531-42b1-93ea-48ca7c06f5e9 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:30.810474 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f174769-7531-42b1-93ea-48ca7c06f5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:30.810798 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f174769-7531-42b1-93ea-48ca7c06f5e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:30.881820 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:30.893952 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f174769-7531-42b1-93ea-48ca7c06f5e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:35:30.895918 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f174769-7531-42b1-93ea-48ca7c06f5e9 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:35:31.012695 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 595/1198] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:35:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 02:35:31.018966 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0e6dde-c3fb-40d2-8664-1465a97749e9 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:31.130615 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0e6dde-c3fb-40d2-8664-1465a97749e9 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:31.166714 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e67614-5c7c-4332-a62c-5c2e81f64e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:31.167299 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e67614-5c7c-4332-a62c-5c2e81f64e06 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:31.167582 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e67614-5c7c-4332-a62c-5c2e81f64e06 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:31.167857 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e67614-5c7c-4332-a62c-5c2e81f64e06 None admin] PATH_INFO: `/v3/projects/5269838069d64b3e9d74d266d13a54be` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:31.169013 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e67614-5c7c-4332-a62c-5c2e81f64e06 None admin] RBAC: Authorizing `identity:get_project(project_id=5269838069d64b3e9d74d266d13a54be)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:35:31.170459 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e67614-5c7c-4332-a62c-5c2e81f64e06 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:35:31.172291 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 604/1199] 10.222.0.20 () {56 vars in 1186 bytes} [Thu Sep 22 02:35:31 2022] GET /identity/v3/projects/5269838069d64b3e9d74d266d13a54be => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:35:31.178847 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f174769-7531-42b1-93ea-48ca7c06f5e9 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:35:31.285092 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:31.294797 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f174769-7531-42b1-93ea-48ca7c06f5e9 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:35:31.330520 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee512e-01cc-4a41-a9f4-42414c0c14b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:35:31.331255 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee512e-01cc-4a41-a9f4-42414c0c14b5 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:35:31.331537 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee512e-01cc-4a41-a9f4-42414c0c14b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:35:31.331810 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee512e-01cc-4a41-a9f4-42414c0c14b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:35:31.332562 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ee512e-01cc-4a41-a9f4-42414c0c14b5 None admin] RBAC: Adding query filter params (role.id=31a687548f55493b8ea666fe7e350303, scope.project.id=5269838069d64b3e9d74d266d13a54be, user.id=784c7ded33d24463a0d870914fc6d339) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:35:31.332874 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ee512e-01cc-4a41-a9f4-42414c0c14b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:35:31.334364 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ee512e-01cc-4a41-a9f4-42414c0c14b5 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:35:31.341131 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 596/1200] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:35:31 2022] GET /identity/v3/role_assignments?user.id=784c7ded33d24463a0d870914fc6d339&scope.project.id=5269838069d64b3e9d74d266d13a54be&role.id=31a687548f55493b8ea666fe7e350303 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:35:41.162319 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=92 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:35:41.338017 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=22 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:29.653441 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d208e81-4904-42ac-bcff-70ebe8f5a5fc None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:29.655309 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d208e81-4904-42ac-bcff-70ebe8f5a5fc None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:29.655309 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d208e81-4904-42ac-bcff-70ebe8f5a5fc None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:29.655309 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 605/1201] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:36:29.664735 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750cb4b-9083-4853-acf2-44d21f5ca6ce None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:29.667346 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750cb4b-9083-4853-acf2-44d21f5ca6ce None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:29.668200 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750cb4b-9083-4853-acf2-44d21f5ca6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:29.702134 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-b750cb4b-9083-4853-acf2-44d21f5ca6ce None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:36:29.704278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-b750cb4b-9083-4853-acf2-44d21f5ca6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:36:29.828118 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 597/1202] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:36:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:36:30.128306 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d208e81-4904-42ac-bcff-70ebe8f5a5fc None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:30.318535 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d208e81-4904-42ac-bcff-70ebe8f5a5fc None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:30.365771 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d823c8f-015b-4d22-b600-58898d656c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:30.366980 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d823c8f-015b-4d22-b600-58898d656c87 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:30.367433 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d823c8f-015b-4d22-b600-58898d656c87 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:30.367869 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d823c8f-015b-4d22-b600-58898d656c87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:30.382778 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d823c8f-015b-4d22-b600-58898d656c87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:30.384641 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d823c8f-015b-4d22-b600-58898d656c87 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:30.409488 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-4d823c8f-015b-4d22-b600-58898d656c87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:36:30.409488 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:36:30.409488 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:36:30.409488 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:36:30.409488 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:36:30.409488 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:36:30.409488 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:36:30.409488 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:36:30.409488 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:36:30.409488 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:36:30.409488 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:36:30.409488 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:36:30.409488 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:36:30.409488 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:36:30.409488 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:36:30.409488 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:36:30.410897 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:36:30.410897 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:36:30.410897 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:36:30.410897 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:36:30.410897 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:36:30.410897 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:36:30.410897 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:36:30.410897 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:36:30.410897 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:36:30.410897 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:36:30.410897 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:36:30.410897 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:36:30.410897 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:36:30.410897 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:36:30.410897 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:36:30.410897 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:36:30.410897 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:36:30.412210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:36:30.412210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:36:30.412210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:36:30.412210 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 606/1203] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:36:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:36:30.422518 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b750cb4b-9083-4853-acf2-44d21f5ca6ce None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:30.549212 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b750cb4b-9083-4853-acf2-44d21f5ca6ce None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:30.591287 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:30.599701 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b173a05e-5e71-41c5-96a4-f514a54cd478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:30.600357 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b173a05e-5e71-41c5-96a4-f514a54cd478 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:30.600657 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b173a05e-5e71-41c5-96a4-f514a54cd478 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:30.600931 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b173a05e-5e71-41c5-96a4-f514a54cd478 None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:30.601590 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b173a05e-5e71-41c5-96a4-f514a54cd478 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:36:30.601909 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b173a05e-5e71-41c5-96a4-f514a54cd478 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:30.603411 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b173a05e-5e71-41c5-96a4-f514a54cd478 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:30.619526 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 598/1204] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:36:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:36:30.626073 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d823c8f-015b-4d22-b600-58898d656c87 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:30.739235 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:30.840363 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d823c8f-015b-4d22-b600-58898d656c87 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:30.911880 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd6670-4cf6-4136-a705-2ac8210a33a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:30.912644 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd6670-4cf6-4136-a705-2ac8210a33a5 None admin] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:30.912988 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd6670-4cf6-4136-a705-2ac8210a33a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:30.913356 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd6670-4cf6-4136-a705-2ac8210a33a5 None admin] PATH_INFO: `/v3/users` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:30.914340 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd6670-4cf6-4136-a705-2ac8210a33a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:30.916280 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd6670-4cf6-4136-a705-2ac8210a33a5 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:30.949716 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 607/1205] 10.222.0.20 () {60 vars in 1132 bytes} [Thu Sep 22 02:36:30 2022] POST /identity/v3/users => generated 251 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:36:33.554366 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b3cfe-5d2e-40cd-ad2b-e72b7d476b3f None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:33.555081 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b3cfe-5d2e-40cd-ad2b-e72b7d476b3f None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:33.555081 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b3cfe-5d2e-40cd-ad2b-e72b7d476b3f None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:33.556621 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 599/1206] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:36:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:36:33.563679 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740cc82c-43ec-46c8-8717-c525f64b945b None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:33.564340 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740cc82c-43ec-46c8-8717-c525f64b945b None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:33.564545 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740cc82c-43ec-46c8-8717-c525f64b945b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:33.587228 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-740cc82c-43ec-46c8-8717-c525f64b945b None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:36:33.588457 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-740cc82c-43ec-46c8-8717-c525f64b945b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:36:33.677147 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 608/1207] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:36:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:36:33.970127 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b3cfe-5d2e-40cd-ad2b-e72b7d476b3f None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:34.148867 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b3cfe-5d2e-40cd-ad2b-e72b7d476b3f None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:34.210775 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fa5dd6-f999-4b20-bd36-0915c48f60af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:34.211472 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa5dd6-f999-4b20-bd36-0915c48f60af None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:34.211759 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa5dd6-f999-4b20-bd36-0915c48f60af None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:34.212023 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa5dd6-f999-4b20-bd36-0915c48f60af None admin] PATH_INFO: `/v3/roles/service` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:34.227994 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fa5dd6-f999-4b20-bd36-0915c48f60af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:36:34.235490 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-75fa5dd6-f999-4b20-bd36-0915c48f60af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:36:34.235490 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:36:34.235490 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:36:34.235490 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:36:34.235490 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:36:34.235490 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:36:34.235490 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:36:34.235490 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:36:34.235490 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:36:34.235490 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:36:34.235490 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:36:34.235490 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:36:34.235490 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:36:34.235490 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:36:34.235490 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:36:34.235490 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:36:34.236597 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:36:34.236597 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:36:34.236597 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:36:34.236597 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:36:34.236597 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:36:34.236597 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:36:34.236597 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:36:34.236597 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:36:34.236597 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:36:34.236597 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:36:34.236597 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:36:34.236597 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:36:34.236597 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:36:34.236597 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:36:34.236597 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:36:34.236597 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:36:34.237489 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:36:34.237489 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:36:34.237489 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:36:34.237489 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:36:34.237489 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:36:34.237489 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:36:34.237489 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:36:34.237489 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:36:34.237489 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:36:34.237489 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:36:34.237489 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:36:34.237489 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:36:34.238286 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 600/1208] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Sep 22 02:36:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:36:34.244391 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740cc82c-43ec-46c8-8717-c525f64b945b None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:34.320648 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:34.405719 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740cc82c-43ec-46c8-8717-c525f64b945b None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:34.450240 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5fb044-ea32-4fd4-8224-53f241b3e67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:34.450902 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5fb044-ea32-4fd4-8224-53f241b3e67c None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:34.451195 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5fb044-ea32-4fd4-8224-53f241b3e67c None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:34.451529 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5fb044-ea32-4fd4-8224-53f241b3e67c None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:34.452326 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5fb044-ea32-4fd4-8224-53f241b3e67c None admin] RBAC: Adding query filter params (name=service) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:36:34.452758 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5fb044-ea32-4fd4-8224-53f241b3e67c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:34.454524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5fb044-ea32-4fd4-8224-53f241b3e67c None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:34.464947 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 609/1209] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Sep 22 02:36:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:36:34.470955 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fa5dd6-f999-4b20-bd36-0915c48f60af None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:34.552730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:34.627855 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fa5dd6-f999-4b20-bd36-0915c48f60af None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:34.682576 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb5a11-20cf-4cf1-a126-c5167a943da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:34.683481 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb5a11-20cf-4cf1-a126-c5167a943da1 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:34.683835 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb5a11-20cf-4cf1-a126-c5167a943da1 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:34.684102 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb5a11-20cf-4cf1-a126-c5167a943da1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:34.698057 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fb5a11-20cf-4cf1-a126-c5167a943da1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:34.700605 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fb5a11-20cf-4cf1-a126-c5167a943da1 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:34.720611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-b8fb5a11-20cf-4cf1-a126-c5167a943da1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:36:34.720611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:36:34.720611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:36:34.720611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:36:34.720611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:36:34.720611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:36:34.720611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:36:34.720611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:36:34.720611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:36:34.720611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:36:34.720611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:36:34.720611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:36:34.720611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:36:34.720611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:36:34.720611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:36:34.720611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:36:34.723027 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:36:34.723027 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:36:34.723027 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:36:34.723027 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:36:34.723027 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:36:34.723027 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:36:34.723027 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:36:34.723027 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:36:34.723027 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:36:34.723027 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:36:34.723027 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:36:34.723027 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:36:34.723027 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:36:34.723027 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:36:34.723027 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:36:34.723027 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:36:34.723027 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:36:34.725090 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:36:34.725090 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:36:34.725090 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:36:34.725090 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 601/1210] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:36:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:36:34.729109 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5fb044-ea32-4fd4-8224-53f241b3e67c None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:34.882543 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:34.886035 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5fb044-ea32-4fd4-8224-53f241b3e67c None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:34.933350 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad75a8-268b-4a4e-88af-f042cebde1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:34.936678 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad75a8-268b-4a4e-88af-f042cebde1c2 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:34.936678 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad75a8-268b-4a4e-88af-f042cebde1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:34.936678 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad75a8-268b-4a4e-88af-f042cebde1c2 None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:34.936678 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad75a8-268b-4a4e-88af-f042cebde1c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:36:34.936678 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad75a8-268b-4a4e-88af-f042cebde1c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:34.940172 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad75a8-268b-4a4e-88af-f042cebde1c2 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:34.968070 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 610/1211] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:36:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:36:34.969807 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb5a11-20cf-4cf1-a126-c5167a943da1 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:35.156857 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb5a11-20cf-4cf1-a126-c5167a943da1 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:35.158752 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:35.208448 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce895a-f24e-4a06-aca7-6986a322c674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:35.209243 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce895a-f24e-4a06-aca7-6986a322c674 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:35.209571 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce895a-f24e-4a06-aca7-6986a322c674 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:35.209861 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce895a-f24e-4a06-aca7-6986a322c674 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:35.219257 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ce895a-f24e-4a06-aca7-6986a322c674 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:35.220740 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ce895a-f24e-4a06-aca7-6986a322c674 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:35.235906 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-96ce895a-f24e-4a06-aca7-6986a322c674 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 22 02:36:35.235906 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:36:35.235906 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:36:35.235906 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:36:35.235906 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:36:35.235906 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:36:35.235906 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:36:35.235906 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:36:35.235906 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:36:35.235906 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:36:35.235906 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:36:35.235906 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:36:35.235906 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:36:35.235906 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:36:35.235906 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:36:35.235906 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:36:35.237012 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:36:35.237012 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:36:35.237012 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:36:35.237012 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:36:35.237012 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:36:35.237012 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:36:35.237012 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:36:35.237012 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:36:35.237012 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:36:35.237012 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:36:35.237012 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:36:35.237012 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:36:35.237012 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:36:35.237012 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:36:35.237012 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:36:35.237012 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:36:35.237012 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:36:35.237012 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:36:35.238370 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:36:35.238370 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:36:35.238370 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:36:35.238370 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:36:35.238370 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:36:35.238370 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:36:35.238370 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:36:35.238370 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:36:35.238370 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 22 02:36:35.238370 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:36:35.238370 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 602/1212] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:36:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:36:35.247007 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad75a8-268b-4a4e-88af-f042cebde1c2 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:35.362416 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad75a8-268b-4a4e-88af-f042cebde1c2 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:35.400728 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58008181-6e8f-4b61-85b8-4d387a82e088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:35.401641 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58008181-6e8f-4b61-85b8-4d387a82e088 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:35.401928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58008181-6e8f-4b61-85b8-4d387a82e088 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:35.402260 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58008181-6e8f-4b61-85b8-4d387a82e088 None admin] PATH_INFO: `/v3/users` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:35.403114 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58008181-6e8f-4b61-85b8-4d387a82e088 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:36:35.403539 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58008181-6e8f-4b61-85b8-4d387a82e088 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:35.405518 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58008181-6e8f-4b61-85b8-4d387a82e088 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:35.433421 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 611/1213] 10.222.0.20 () {56 vars in 1143 bytes} [Thu Sep 22 02:36:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:36:35.440918 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce895a-f24e-4a06-aca7-6986a322c674 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:35.584298 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce895a-f24e-4a06-aca7-6986a322c674 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:35.637130 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3f929-c280-478b-9cd8-afd5e97122bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:35.637787 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3f929-c280-478b-9cd8-afd5e97122bc None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:35.638232 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3f929-c280-478b-9cd8-afd5e97122bc None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:35.638522 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3f929-c280-478b-9cd8-afd5e97122bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:35.681906 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:35.737231 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea3f929-c280-478b-9cd8-afd5e97122bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:35.738598 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea3f929-c280-478b-9cd8-afd5e97122bc None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:35.919732 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 603/1214] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:36:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 22 02:36:35.921746 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58008181-6e8f-4b61-85b8-4d387a82e088 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:35.948113 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:36.058805 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58008181-6e8f-4b61-85b8-4d387a82e088 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:36.110875 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aacb28-3632-47c0-aba1-a42902312a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:36.111801 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aacb28-3632-47c0-aba1-a42902312a6e None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:36.112183 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aacb28-3632-47c0-aba1-a42902312a6e None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:36.112575 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aacb28-3632-47c0-aba1-a42902312a6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:36.130577 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aacb28-3632-47c0-aba1-a42902312a6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:36.133835 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aacb28-3632-47c0-aba1-a42902312a6e None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:36.163298 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-40aacb28-3632-47c0-aba1-a42902312a6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:36:36.163298 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:36:36.163298 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:36:36.163298 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:36:36.163298 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:36:36.163298 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:36:36.163298 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:36:36.163298 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:36:36.163298 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:36:36.163298 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:36:36.163298 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:36:36.163298 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:36:36.163298 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:36:36.163298 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:36:36.163298 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:36:36.163298 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:36:36.173365 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:36:36.173365 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:36:36.173365 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:36:36.173365 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:36:36.173365 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:36:36.173365 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:36:36.173365 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:36:36.173365 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:36:36.173365 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:36:36.173365 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:36:36.173365 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:36:36.173365 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:36:36.173365 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:36:36.173365 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:36:36.173365 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:36:36.173365 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:36:36.173365 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:36:36.175045 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:36:36.175045 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:36:36.175045 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:36:36.175045 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 612/1215] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:36:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:36:36.175365 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3f929-c280-478b-9cd8-afd5e97122bc None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:36.280779 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:36.365706 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3f929-c280-478b-9cd8-afd5e97122bc None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:36.409888 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19373ba4-f30f-464d-bf4f-1d685c193271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:36.410741 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19373ba4-f30f-464d-bf4f-1d685c193271 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:36.411182 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19373ba4-f30f-464d-bf4f-1d685c193271 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:36.411455 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19373ba4-f30f-464d-bf4f-1d685c193271 None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:36.412535 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19373ba4-f30f-464d-bf4f-1d685c193271 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:36:36.412976 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19373ba4-f30f-464d-bf4f-1d685c193271 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:36.414610 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19373ba4-f30f-464d-bf4f-1d685c193271 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:36.432123 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 604/1216] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:36:36.439351 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aacb28-3632-47c0-aba1-a42902312a6e None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:36.551923 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:36.593317 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aacb28-3632-47c0-aba1-a42902312a6e None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:36.632920 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08548d8-35bc-45c0-baad-0f9970df14b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:36.633691 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08548d8-35bc-45c0-baad-0f9970df14b2 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:36.634047 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08548d8-35bc-45c0-baad-0f9970df14b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:36.634354 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08548d8-35bc-45c0-baad-0f9970df14b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:36.642981 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08548d8-35bc-45c0-baad-0f9970df14b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:36.644915 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08548d8-35bc-45c0-baad-0f9970df14b2 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:36.662724 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-b08548d8-35bc-45c0-baad-0f9970df14b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:36:36.662724 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:36:36.662724 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:36:36.662724 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:36:36.662724 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:36:36.662724 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:36:36.662724 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:36:36.662724 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:36:36.662724 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:36:36.662724 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:36:36.662724 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:36:36.662724 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:36:36.662724 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:36:36.662724 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:36:36.662724 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:36:36.664640 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:36:36.664640 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:36:36.664640 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:36:36.664640 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:36:36.664640 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:36:36.664640 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:36:36.664640 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:36:36.664640 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:36:36.664640 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:36:36.664640 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:36:36.664640 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:36:36.664640 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:36:36.664640 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:36:36.664640 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:36:36.664640 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:36:36.664640 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:36:36.664640 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:36:36.666444 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:36:36.666444 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:36:36.666444 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:36:36.666444 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:36:36.666444 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:36:36.666444 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:36:36.666444 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:36:36.666444 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:36:36.666444 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 613/1217] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:36:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:36:36.673198 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19373ba4-f30f-464d-bf4f-1d685c193271 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:36.806448 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19373ba4-f30f-464d-bf4f-1d685c193271 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:36.853342 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:36.875050 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ab1880-3e21-4df3-82a6-337bf5e030c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:36.876101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ab1880-3e21-4df3-82a6-337bf5e030c4 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:36.876558 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ab1880-3e21-4df3-82a6-337bf5e030c4 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:36.877055 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ab1880-3e21-4df3-82a6-337bf5e030c4 None admin] PATH_INFO: `/v3/projects` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:36.878196 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ab1880-3e21-4df3-82a6-337bf5e030c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:36:36.878755 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ab1880-3e21-4df3-82a6-337bf5e030c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:36.881535 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ab1880-3e21-4df3-82a6-337bf5e030c4 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:36.909333 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 605/1218] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Sep 22 02:36:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:36:36.917079 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08548d8-35bc-45c0-baad-0f9970df14b2 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:37.073013 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08548d8-35bc-45c0-baad-0f9970df14b2 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:37.115283 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:37.125294 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d585b-ed68-4f65-bc69-1bf6c4b64942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:37.126178 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d585b-ed68-4f65-bc69-1bf6c4b64942 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:37.126560 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d585b-ed68-4f65-bc69-1bf6c4b64942 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:37.127022 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d585b-ed68-4f65-bc69-1bf6c4b64942 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:37.128067 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93d585b-ed68-4f65-bc69-1bf6c4b64942 None admin] RBAC: Adding query filter params (role.id=fb2fbee669e14ffbbe0aa70642eab1ce, scope.project.id=2a792f8dc45145d687ddd58c94d44b5e, user.id=865576e9e17d4efdbd1ca6a005f31bda) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:36:37.128507 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93d585b-ed68-4f65-bc69-1bf6c4b64942 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:37.130511 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93d585b-ed68-4f65-bc69-1bf6c4b64942 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:37.139956 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 614/1219] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:36:36 2022] GET /identity/v3/role_assignments?user.id=865576e9e17d4efdbd1ca6a005f31bda&scope.project.id=2a792f8dc45145d687ddd58c94d44b5e&role.id=fb2fbee669e14ffbbe0aa70642eab1ce => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:36:39.641660 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35546a58-aa4c-4d68-bb73-03ba680a6d52 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:39.642269 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35546a58-aa4c-4d68-bb73-03ba680a6d52 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:39.642269 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35546a58-aa4c-4d68-bb73-03ba680a6d52 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:39.643680 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 606/1220] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:36:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:36:39.657278 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753bf5e-a585-4cca-baf5-78f37228baec None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:39.657840 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753bf5e-a585-4cca-baf5-78f37228baec None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:39.657840 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753bf5e-a585-4cca-baf5-78f37228baec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:39.680524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-3753bf5e-a585-4cca-baf5-78f37228baec None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:36:39.682028 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-3753bf5e-a585-4cca-baf5-78f37228baec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:36:39.781397 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 615/1221] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:36:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:36:40.052670 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35546a58-aa4c-4d68-bb73-03ba680a6d52 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:40.174656 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35546a58-aa4c-4d68-bb73-03ba680a6d52 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:40.224960 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d235a20-7f1c-40fa-a570-6f85b3a9c6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:40.225757 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d235a20-7f1c-40fa-a570-6f85b3a9c6b4 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:40.226134 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d235a20-7f1c-40fa-a570-6f85b3a9c6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:40.226407 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d235a20-7f1c-40fa-a570-6f85b3a9c6b4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:40.240689 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d235a20-7f1c-40fa-a570-6f85b3a9c6b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:36:40.248689 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-0d235a20-7f1c-40fa-a570-6f85b3a9c6b4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:36:40.248689 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:36:40.248689 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:36:40.248689 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:36:40.248689 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:36:40.248689 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:36:40.248689 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:36:40.248689 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:36:40.248689 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:36:40.248689 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:36:40.248689 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:36:40.248689 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:36:40.248689 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:36:40.248689 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:36:40.248689 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:36:40.248689 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:36:40.250153 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:36:40.250153 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:36:40.250153 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:36:40.250153 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:36:40.250153 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:36:40.250153 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:36:40.250153 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:36:40.250153 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:36:40.250153 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:36:40.250153 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:36:40.250153 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:36:40.250153 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:36:40.250153 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:36:40.250153 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:36:40.250153 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:36:40.250153 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:36:40.251101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:36:40.251101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:36:40.251101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:36:40.251101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:36:40.251101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:36:40.251101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:36:40.251101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:36:40.251101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:36:40.251101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:36:40.251101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:36:40.251101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:36:40.251101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:36:40.251101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 607/1222] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Sep 22 02:36:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:36:40.257573 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753bf5e-a585-4cca-baf5-78f37228baec None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:40.415996 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753bf5e-a585-4cca-baf5-78f37228baec None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:40.456230 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:40.473374 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a76dff-695d-48ee-b80a-e61f78520ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:40.474148 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a76dff-695d-48ee-b80a-e61f78520ce7 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:40.474467 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a76dff-695d-48ee-b80a-e61f78520ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:40.474918 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a76dff-695d-48ee-b80a-e61f78520ce7 None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:40.475814 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a76dff-695d-48ee-b80a-e61f78520ce7 None admin] RBAC: Adding query filter params (name=service) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:36:40.476123 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a76dff-695d-48ee-b80a-e61f78520ce7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:40.478007 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a76dff-695d-48ee-b80a-e61f78520ce7 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:40.489671 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 616/1223] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Sep 22 02:36:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:36:40.497257 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d235a20-7f1c-40fa-a570-6f85b3a9c6b4 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:40.576216 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:40.658927 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d235a20-7f1c-40fa-a570-6f85b3a9c6b4 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:40.712939 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3bd3e-0252-44ea-983b-98dcb9f9c9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:40.713994 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3bd3e-0252-44ea-983b-98dcb9f9c9b0 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:40.714600 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3bd3e-0252-44ea-983b-98dcb9f9c9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:40.715034 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3bd3e-0252-44ea-983b-98dcb9f9c9b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:40.728249 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d3bd3e-0252-44ea-983b-98dcb9f9c9b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:40.730276 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d3bd3e-0252-44ea-983b-98dcb9f9c9b0 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:40.746935 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-28d3bd3e-0252-44ea-983b-98dcb9f9c9b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:36:40.746935 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:36:40.746935 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:36:40.746935 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:36:40.746935 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:36:40.746935 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:36:40.746935 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:36:40.746935 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:36:40.746935 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:36:40.746935 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:36:40.746935 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:36:40.746935 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:36:40.746935 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:36:40.746935 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:36:40.746935 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:36:40.746935 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:36:40.748989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:36:40.748989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:36:40.748989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:36:40.748989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:36:40.748989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:36:40.748989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:36:40.748989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:36:40.748989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:36:40.748989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:36:40.748989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:36:40.748989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:36:40.748989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:36:40.748989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:36:40.748989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:36:40.748989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:36:40.748989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:36:40.748989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:36:40.751024 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:36:40.751024 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:36:40.751024 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:36:40.751024 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 608/1224] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:36:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:36:40.755779 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a76dff-695d-48ee-b80a-e61f78520ce7 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:40.914499 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a76dff-695d-48ee-b80a-e61f78520ce7 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:40.968351 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49141bb6-fbc8-4d81-9def-0c7fbcff248a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:40.968960 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49141bb6-fbc8-4d81-9def-0c7fbcff248a None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:40.969503 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49141bb6-fbc8-4d81-9def-0c7fbcff248a None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:40.969884 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49141bb6-fbc8-4d81-9def-0c7fbcff248a None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:40.970869 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49141bb6-fbc8-4d81-9def-0c7fbcff248a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:36:40.971309 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49141bb6-fbc8-4d81-9def-0c7fbcff248a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:40.973360 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49141bb6-fbc8-4d81-9def-0c7fbcff248a None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:40.990297 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 617/1225] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:36:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:36:40.996439 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3bd3e-0252-44ea-983b-98dcb9f9c9b0 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:41.124905 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:41.133510 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3bd3e-0252-44ea-983b-98dcb9f9c9b0 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:41.186476 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6b185-c096-4c56-8b01-e778a3dd47f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:41.187641 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6b185-c096-4c56-8b01-e778a3dd47f5 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:41.188167 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6b185-c096-4c56-8b01-e778a3dd47f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:41.188595 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6b185-c096-4c56-8b01-e778a3dd47f5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:41.198790 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee6b185-c096-4c56-8b01-e778a3dd47f5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:41.200487 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee6b185-c096-4c56-8b01-e778a3dd47f5 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:41.215101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-eee6b185-c096-4c56-8b01-e778a3dd47f5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 22 02:36:41.215101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:36:41.215101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:36:41.215101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:36:41.215101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:36:41.215101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:36:41.215101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:36:41.215101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:36:41.215101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:36:41.215101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:36:41.215101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:36:41.215101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:36:41.215101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:36:41.215101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:36:41.215101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:36:41.215101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:36:41.217412 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:36:41.217412 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:36:41.217412 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:36:41.217412 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:36:41.217412 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:36:41.217412 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:36:41.217412 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:36:41.217412 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:36:41.217412 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:36:41.217412 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:36:41.217412 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:36:41.217412 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:36:41.217412 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:36:41.217412 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:36:41.217412 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:36:41.217412 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:36:41.217412 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:36:41.217412 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:36:41.219358 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:36:41.219358 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:36:41.219358 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:36:41.219358 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:36:41.219358 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:36:41.219358 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:36:41.219358 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:36:41.219358 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:36:41.219358 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 22 02:36:41.219358 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:36:41.219358 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 609/1226] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:36:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:36:41.223514 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49141bb6-fbc8-4d81-9def-0c7fbcff248a None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:41.282327 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:41.360010 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49141bb6-fbc8-4d81-9def-0c7fbcff248a None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:41.415319 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39050414-c126-4efe-84c0-2117a9511162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:41.416346 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39050414-c126-4efe-84c0-2117a9511162 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:41.416717 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39050414-c126-4efe-84c0-2117a9511162 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:41.417545 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39050414-c126-4efe-84c0-2117a9511162 None admin] PATH_INFO: `/v3/users` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:41.418358 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39050414-c126-4efe-84c0-2117a9511162 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:36:41.418715 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39050414-c126-4efe-84c0-2117a9511162 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:41.421533 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39050414-c126-4efe-84c0-2117a9511162 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:41.446166 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 618/1227] 10.222.0.20 () {56 vars in 1143 bytes} [Thu Sep 22 02:36:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:36:41.452295 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6b185-c096-4c56-8b01-e778a3dd47f5 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:41.592783 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6b185-c096-4c56-8b01-e778a3dd47f5 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:41.640803 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870347f4-8482-470e-ad32-794d0e55df65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:41.640803 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870347f4-8482-470e-ad32-794d0e55df65 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:41.640803 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870347f4-8482-470e-ad32-794d0e55df65 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:41.640803 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870347f4-8482-470e-ad32-794d0e55df65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:41.655968 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870347f4-8482-470e-ad32-794d0e55df65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:41.657512 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870347f4-8482-470e-ad32-794d0e55df65 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:41.680514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-870347f4-8482-470e-ad32-794d0e55df65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:36:41.680514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:36:41.680514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:36:41.680514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:36:41.680514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:36:41.680514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:36:41.680514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:36:41.680514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:36:41.680514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:36:41.680514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:36:41.680514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:36:41.680514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:36:41.680514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:36:41.680514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:36:41.680514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:36:41.680514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:36:41.682303 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:36:41.682303 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:36:41.682303 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:36:41.682303 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:36:41.682303 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:36:41.682303 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:36:41.682303 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:36:41.682303 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:36:41.682303 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:36:41.682303 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:36:41.682303 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:36:41.682303 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:36:41.682303 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:36:41.682303 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:36:41.682303 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:36:41.682303 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:36:41.682303 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:36:41.683640 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:36:41.683640 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:36:41.683640 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:36:41.683640 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 610/1228] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:36:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:36:41.688883 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39050414-c126-4efe-84c0-2117a9511162 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:41.829494 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39050414-c126-4efe-84c0-2117a9511162 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:41.832279 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:41.873886 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b98f28c-04cc-4a11-8b4c-306f8bccb2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:41.874772 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b98f28c-04cc-4a11-8b4c-306f8bccb2f8 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:41.875636 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b98f28c-04cc-4a11-8b4c-306f8bccb2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:41.876164 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b98f28c-04cc-4a11-8b4c-306f8bccb2f8 None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:41.877293 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b98f28c-04cc-4a11-8b4c-306f8bccb2f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:36:41.877719 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b98f28c-04cc-4a11-8b4c-306f8bccb2f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:41.879608 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b98f28c-04cc-4a11-8b4c-306f8bccb2f8 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:41.902582 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 619/1229] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:36:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:36:41.905525 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870347f4-8482-470e-ad32-794d0e55df65 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:41.929101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:42.054603 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870347f4-8482-470e-ad32-794d0e55df65 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:42.095623 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad84eec-4682-4b53-8e5e-14551b0bdc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:42.096317 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad84eec-4682-4b53-8e5e-14551b0bdc5a None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:42.096596 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad84eec-4682-4b53-8e5e-14551b0bdc5a None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:42.097055 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad84eec-4682-4b53-8e5e-14551b0bdc5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:42.105094 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad84eec-4682-4b53-8e5e-14551b0bdc5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:42.106578 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad84eec-4682-4b53-8e5e-14551b0bdc5a None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:42.119557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-fad84eec-4682-4b53-8e5e-14551b0bdc5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:36:42.119557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:36:42.119557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:36:42.119557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:36:42.119557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:36:42.119557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:36:42.119557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:36:42.119557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:36:42.119557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:36:42.119557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:36:42.119557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:36:42.119557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:36:42.119557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:36:42.119557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:36:42.119557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:36:42.121485 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:36:42.121485 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:36:42.121485 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:36:42.121485 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:36:42.121485 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:36:42.121485 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:36:42.121485 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:36:42.121485 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:36:42.121485 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:36:42.121485 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:36:42.121485 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:36:42.121485 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:36:42.121485 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:36:42.121485 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:36:42.121485 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:36:42.121485 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:36:42.121485 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:36:42.123439 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:36:42.123439 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:36:42.123439 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:36:42.123439 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:36:42.123439 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:36:42.123439 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:36:42.123439 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:36:42.123439 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:36:42.123439 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 611/1230] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:36:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:36:42.127658 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b98f28c-04cc-4a11-8b4c-306f8bccb2f8 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:42.248333 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b98f28c-04cc-4a11-8b4c-306f8bccb2f8 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:42.295243 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7610585c-e37b-4e5f-9e04-373de3b6081b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:42.296087 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7610585c-e37b-4e5f-9e04-373de3b6081b None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:42.296481 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7610585c-e37b-4e5f-9e04-373de3b6081b None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:42.296823 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7610585c-e37b-4e5f-9e04-373de3b6081b None admin] PATH_INFO: `/v3/projects` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:42.297594 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7610585c-e37b-4e5f-9e04-373de3b6081b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:36:42.297907 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7610585c-e37b-4e5f-9e04-373de3b6081b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:42.299430 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7610585c-e37b-4e5f-9e04-373de3b6081b None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:42.313676 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 620/1231] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Sep 22 02:36:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:36:42.320334 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad84eec-4682-4b53-8e5e-14551b0bdc5a None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:42.402655 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:42.457833 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad84eec-4682-4b53-8e5e-14551b0bdc5a None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:42.498408 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e5ae29-2f19-460f-acf7-54a77d26a3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:42.499113 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5ae29-2f19-460f-acf7-54a77d26a3f8 None admin] REQUEST_METHOD: `PUT` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:42.499404 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5ae29-2f19-460f-acf7-54a77d26a3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:42.499696 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5ae29-2f19-460f-acf7-54a77d26a3f8 None admin] PATH_INFO: `/v3/projects/2a792f8dc45145d687ddd58c94d44b5e/users/865576e9e17d4efdbd1ca6a005f31bda/roles/fb2fbee669e14ffbbe0aa70642eab1ce` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:42.523140 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e5ae29-2f19-460f-acf7-54a77d26a3f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a792f8dc45145d687ddd58c94d44b5e, user_id=865576e9e17d4efdbd1ca6a005f31bda, role_id=fb2fbee669e14ffbbe0aa70642eab1ce)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:42.525751 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e5ae29-2f19-460f-acf7-54a77d26a3f8 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:42.542994 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 612/1232] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Sep 22 02:36:42 2022] PUT /identity/v3/projects/2a792f8dc45145d687ddd58c94d44b5e/users/865576e9e17d4efdbd1ca6a005f31bda/roles/fb2fbee669e14ffbbe0aa70642eab1ce => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 02:36:45.008001 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4deac50-5a87-4db2-b557-32e2cc09e8f0 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:45.009205 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4deac50-5a87-4db2-b557-32e2cc09e8f0 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:45.009205 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4deac50-5a87-4db2-b557-32e2cc09e8f0 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:45.010537 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 621/1233] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:36:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:36:45.016275 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450275f-31e2-44e9-8cf6-42ae01655956 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:45.016853 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450275f-31e2-44e9-8cf6-42ae01655956 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:45.016853 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450275f-31e2-44e9-8cf6-42ae01655956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:45.038181 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-7450275f-31e2-44e9-8cf6-42ae01655956 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:36:45.039543 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-7450275f-31e2-44e9-8cf6-42ae01655956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:36:45.145113 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 613/1234] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:36:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:36:45.432882 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4deac50-5a87-4db2-b557-32e2cc09e8f0 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:45.463921 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:45.587469 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4deac50-5a87-4db2-b557-32e2cc09e8f0 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:45.644452 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b922196-4414-411b-8595-83a348ec44fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:45.645782 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b922196-4414-411b-8595-83a348ec44fd None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:45.646513 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b922196-4414-411b-8595-83a348ec44fd None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:45.647103 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b922196-4414-411b-8595-83a348ec44fd None admin] PATH_INFO: `/v3/roles/service` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:45.667181 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b922196-4414-411b-8595-83a348ec44fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:36:45.674237 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-1b922196-4414-411b-8595-83a348ec44fd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:36:45.674237 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:36:45.674237 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:36:45.674237 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:36:45.674237 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:36:45.674237 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:36:45.674237 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:36:45.674237 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:36:45.674237 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:36:45.674237 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:36:45.674237 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:36:45.674237 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:36:45.674237 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:36:45.674237 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:36:45.674237 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:36:45.674237 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:36:45.676338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:36:45.676338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:36:45.676338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:36:45.676338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:36:45.676338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:36:45.676338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:36:45.676338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:36:45.676338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:36:45.676338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:36:45.676338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:36:45.676338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:36:45.676338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:36:45.676338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:36:45.676338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:36:45.676338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:36:45.676338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:36:45.677392 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:36:45.677392 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:36:45.677392 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:36:45.677392 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:36:45.677392 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:36:45.677392 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:36:45.677392 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:36:45.677392 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:36:45.677392 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:36:45.677392 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:36:45.677392 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 02:36:45.677392 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:36:45.677392 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 622/1235] 10.222.0.20 () {56 vars in 1102 bytes} [Thu Sep 22 02:36:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:36:45.686151 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450275f-31e2-44e9-8cf6-42ae01655956 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:45.764623 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:45.842791 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450275f-31e2-44e9-8cf6-42ae01655956 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:45.882771 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b00aa2-3003-4c3b-b9cd-66f93478b115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:45.883473 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b00aa2-3003-4c3b-b9cd-66f93478b115 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:45.883780 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b00aa2-3003-4c3b-b9cd-66f93478b115 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:45.884187 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b00aa2-3003-4c3b-b9cd-66f93478b115 None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:45.885030 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b00aa2-3003-4c3b-b9cd-66f93478b115 None admin] RBAC: Adding query filter params (name=service) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:36:45.886076 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b00aa2-3003-4c3b-b9cd-66f93478b115 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:45.888053 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b00aa2-3003-4c3b-b9cd-66f93478b115 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:45.906098 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 614/1236] 10.222.0.20 () {56 vars in 1103 bytes} [Thu Sep 22 02:36:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:36:45.914543 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b922196-4414-411b-8595-83a348ec44fd None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:46.027164 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:46.054359 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b922196-4414-411b-8595-83a348ec44fd None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:46.110901 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af6c45-191b-4ad9-9833-36d38c5d7eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:46.111944 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af6c45-191b-4ad9-9833-36d38c5d7eba None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:46.112496 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af6c45-191b-4ad9-9833-36d38c5d7eba None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:46.112858 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af6c45-191b-4ad9-9833-36d38c5d7eba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:46.129713 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01af6c45-191b-4ad9-9833-36d38c5d7eba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:46.131967 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01af6c45-191b-4ad9-9833-36d38c5d7eba None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:46.156783 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-01af6c45-191b-4ad9-9833-36d38c5d7eba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:36:46.156783 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:36:46.156783 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:36:46.156783 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:36:46.156783 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:36:46.156783 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:36:46.156783 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:36:46.156783 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:36:46.156783 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:36:46.156783 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:36:46.156783 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:36:46.156783 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:36:46.156783 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:36:46.156783 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:36:46.156783 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:36:46.156783 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:36:46.157912 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:36:46.157912 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:36:46.157912 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:36:46.157912 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:36:46.157912 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:36:46.157912 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:36:46.157912 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:36:46.157912 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:36:46.157912 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:36:46.157912 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:36:46.157912 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:36:46.157912 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:36:46.157912 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:36:46.157912 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:36:46.157912 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:36:46.157912 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:36:46.157912 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:36:46.158942 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:36:46.158942 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:36:46.158942 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:36:46.158942 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 623/1237] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:36:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:36:46.168322 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b00aa2-3003-4c3b-b9cd-66f93478b115 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:46.311726 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b00aa2-3003-4c3b-b9cd-66f93478b115 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:46.322272 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:46.359747 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e9b437-2777-49d9-bd8d-bad93a5c578c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:46.360978 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9b437-2777-49d9-bd8d-bad93a5c578c None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:46.361621 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9b437-2777-49d9-bd8d-bad93a5c578c None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:46.361913 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9b437-2777-49d9-bd8d-bad93a5c578c None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:46.363308 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e9b437-2777-49d9-bd8d-bad93a5c578c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:36:46.364016 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e9b437-2777-49d9-bd8d-bad93a5c578c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:46.366238 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e9b437-2777-49d9-bd8d-bad93a5c578c None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:46.388819 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 615/1238] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:36:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:36:46.395126 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af6c45-191b-4ad9-9833-36d38c5d7eba None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:46.536514 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af6c45-191b-4ad9-9833-36d38c5d7eba None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:46.592958 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6ed61-004f-401e-bbfe-9784ce7613bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:46.593618 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6ed61-004f-401e-bbfe-9784ce7613bf None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:46.593902 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6ed61-004f-401e-bbfe-9784ce7613bf None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:46.594173 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6ed61-004f-401e-bbfe-9784ce7613bf None admin] PATH_INFO: `/v3/users/placement` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:46.597650 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:46.604675 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea6ed61-004f-401e-bbfe-9784ce7613bf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:46.606306 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea6ed61-004f-401e-bbfe-9784ce7613bf None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:46.618518 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-fea6ed61-004f-401e-bbfe-9784ce7613bf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 22 02:36:46.618518 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:36:46.618518 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:36:46.618518 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:36:46.618518 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:36:46.618518 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:36:46.618518 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:36:46.618518 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:36:46.618518 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:36:46.618518 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:36:46.618518 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:36:46.618518 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:36:46.618518 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:36:46.618518 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:36:46.618518 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:36:46.618518 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:36:46.621334 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:36:46.621334 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:36:46.621334 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:36:46.621334 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:36:46.621334 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:36:46.621334 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:36:46.621334 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:36:46.621334 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:36:46.621334 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:36:46.621334 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:36:46.621334 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:36:46.621334 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:36:46.621334 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:36:46.621334 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:36:46.621334 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:36:46.621334 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:36:46.621334 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:36:46.621334 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:36:46.623288 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:36:46.623288 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:36:46.623288 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:36:46.623288 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:36:46.623288 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:36:46.623288 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:36:46.623288 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:36:46.623288 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:36:46.623288 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 22 02:36:46.623288 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:36:46.623288 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 624/1239] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:36:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:36:46.628386 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e9b437-2777-49d9-bd8d-bad93a5c578c None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:46.781022 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e9b437-2777-49d9-bd8d-bad93a5c578c None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:46.832668 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96440ed-6ee9-4e9b-a002-7e59e5834554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:46.833657 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96440ed-6ee9-4e9b-a002-7e59e5834554 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:46.834131 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96440ed-6ee9-4e9b-a002-7e59e5834554 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:46.834572 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96440ed-6ee9-4e9b-a002-7e59e5834554 None admin] PATH_INFO: `/v3/users` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:46.835943 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96440ed-6ee9-4e9b-a002-7e59e5834554 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:36:46.836344 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96440ed-6ee9-4e9b-a002-7e59e5834554 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:46.838813 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96440ed-6ee9-4e9b-a002-7e59e5834554 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:46.863020 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 616/1240] 10.222.0.20 () {56 vars in 1143 bytes} [Thu Sep 22 02:36:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:36:46.868848 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6ed61-004f-401e-bbfe-9784ce7613bf None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:46.998990 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6ed61-004f-401e-bbfe-9784ce7613bf None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:47.041600 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f7bf8-4f2e-40c0-94b9-f605834a7116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:47.042926 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f7bf8-4f2e-40c0-94b9-f605834a7116 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:47.043256 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f7bf8-4f2e-40c0-94b9-f605834a7116 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:47.043621 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f7bf8-4f2e-40c0-94b9-f605834a7116 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:47.124417 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:47.135567 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6f7bf8-4f2e-40c0-94b9-f605834a7116 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:47.137837 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6f7bf8-4f2e-40c0-94b9-f605834a7116 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:47.287790 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 625/1241] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:36:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 22 02:36:47.289166 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96440ed-6ee9-4e9b-a002-7e59e5834554 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:47.327805 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:47.422869 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96440ed-6ee9-4e9b-a002-7e59e5834554 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:47.468774 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8140806-6d69-4d80-9d1a-fe8739a0ce80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:47.469642 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8140806-6d69-4d80-9d1a-fe8739a0ce80 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:47.470055 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8140806-6d69-4d80-9d1a-fe8739a0ce80 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:47.470457 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8140806-6d69-4d80-9d1a-fe8739a0ce80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:47.485511 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8140806-6d69-4d80-9d1a-fe8739a0ce80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:47.487186 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8140806-6d69-4d80-9d1a-fe8739a0ce80 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:47.506373 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-a8140806-6d69-4d80-9d1a-fe8739a0ce80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:36:47.506373 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:36:47.506373 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:36:47.506373 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:36:47.506373 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:36:47.506373 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:36:47.506373 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:36:47.506373 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:36:47.506373 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:36:47.506373 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:36:47.506373 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:36:47.506373 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:36:47.506373 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:36:47.506373 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:36:47.506373 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:36:47.506373 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:36:47.508528 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:36:47.508528 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:36:47.508528 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:36:47.508528 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:36:47.508528 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:36:47.508528 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:36:47.508528 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:36:47.508528 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:36:47.508528 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:36:47.508528 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:36:47.508528 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:36:47.508528 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:36:47.508528 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:36:47.508528 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:36:47.508528 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:36:47.508528 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:36:47.508528 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:36:47.509799 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:36:47.509799 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:36:47.509799 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:36:47.509799 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 617/1242] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:36:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:36:47.517678 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f7bf8-4f2e-40c0-94b9-f605834a7116 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:47.679222 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:47.688232 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f7bf8-4f2e-40c0-94b9-f605834a7116 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:47.733158 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a2326-13cb-4a6e-9699-80b788df3832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:47.733959 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a2326-13cb-4a6e-9699-80b788df3832 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:47.734313 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a2326-13cb-4a6e-9699-80b788df3832 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:47.734664 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a2326-13cb-4a6e-9699-80b788df3832 None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:47.735562 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9a2326-13cb-4a6e-9699-80b788df3832 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:36:47.735895 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9a2326-13cb-4a6e-9699-80b788df3832 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:47.737458 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9a2326-13cb-4a6e-9699-80b788df3832 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:47.751775 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 626/1243] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:36:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:36:47.758254 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8140806-6d69-4d80-9d1a-fe8739a0ce80 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:47.886461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:47.918540 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8140806-6d69-4d80-9d1a-fe8739a0ce80 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:47.969998 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52dc3ed-2135-47f9-8dd5-99d91956718a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:47.970891 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52dc3ed-2135-47f9-8dd5-99d91956718a None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:47.971380 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52dc3ed-2135-47f9-8dd5-99d91956718a None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:47.971878 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52dc3ed-2135-47f9-8dd5-99d91956718a None admin] PATH_INFO: `/v3/projects/service` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:47.983252 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52dc3ed-2135-47f9-8dd5-99d91956718a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:47.985190 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52dc3ed-2135-47f9-8dd5-99d91956718a None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:48.000168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-a52dc3ed-2135-47f9-8dd5-99d91956718a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:36:48.000168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:36:48.000168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:36:48.000168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:36:48.000168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:36:48.000168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:36:48.000168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:36:48.000168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:36:48.000168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:36:48.000168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:36:48.000168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:36:48.000168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:36:48.000168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:36:48.000168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:36:48.000168 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:36:48.004035 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:36:48.004035 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:36:48.004035 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:36:48.004035 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:36:48.004035 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:36:48.004035 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:36:48.004035 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:36:48.004035 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:36:48.004035 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:36:48.004035 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:36:48.004035 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:36:48.004035 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:36:48.004035 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:36:48.004035 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:36:48.004035 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:36:48.004035 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:36:48.004035 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:36:48.005406 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:36:48.005406 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:36:48.005406 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:36:48.005406 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:36:48.005406 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:36:48.005406 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:36:48.005406 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:36:48.005406 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:36:48.006470 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 618/1244] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:36:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:36:48.007827 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a2326-13cb-4a6e-9699-80b788df3832 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:48.144289 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a2326-13cb-4a6e-9699-80b788df3832 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:48.195805 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b129a4ff-0d74-4af7-8ca0-5aa4cc4fde29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:48.196602 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129a4ff-0d74-4af7-8ca0-5aa4cc4fde29 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:48.196891 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129a4ff-0d74-4af7-8ca0-5aa4cc4fde29 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:48.197172 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129a4ff-0d74-4af7-8ca0-5aa4cc4fde29 None admin] PATH_INFO: `/v3/projects` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:48.198162 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b129a4ff-0d74-4af7-8ca0-5aa4cc4fde29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:36:48.198494 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b129a4ff-0d74-4af7-8ca0-5aa4cc4fde29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:48.200211 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b129a4ff-0d74-4af7-8ca0-5aa4cc4fde29 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:48.214686 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 627/1245] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Sep 22 02:36:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:36:48.222129 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52dc3ed-2135-47f9-8dd5-99d91956718a None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:48.362665 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52dc3ed-2135-47f9-8dd5-99d91956718a None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:48.404354 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92d1fc7-e139-405d-b905-e729dd8d54be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:48.405030 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d1fc7-e139-405d-b905-e729dd8d54be None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:48.405291 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d1fc7-e139-405d-b905-e729dd8d54be None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:48.405676 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d1fc7-e139-405d-b905-e729dd8d54be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:48.406506 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92d1fc7-e139-405d-b905-e729dd8d54be None admin] RBAC: Adding query filter params (role.id=fb2fbee669e14ffbbe0aa70642eab1ce, scope.project.id=2a792f8dc45145d687ddd58c94d44b5e, user.id=865576e9e17d4efdbd1ca6a005f31bda) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:36:48.406864 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92d1fc7-e139-405d-b905-e729dd8d54be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:48.408473 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92d1fc7-e139-405d-b905-e729dd8d54be None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:48.415579 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 619/1246] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:36:48 2022] GET /identity/v3/role_assignments?user.id=865576e9e17d4efdbd1ca6a005f31bda&scope.project.id=2a792f8dc45145d687ddd58c94d44b5e&role.id=fb2fbee669e14ffbbe0aa70642eab1ce => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:36:50.814407 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cfc023-da24-41c7-ba45-b09ade56d725 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:50.815051 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cfc023-da24-41c7-ba45-b09ade56d725 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:50.815168 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cfc023-da24-41c7-ba45-b09ade56d725 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:50.816460 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 628/1247] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:36:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:36:50.822621 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248d79e-e4a6-4108-b840-d5911c52a7d4 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:50.823149 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248d79e-e4a6-4108-b840-d5911c52a7d4 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:50.823230 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248d79e-e4a6-4108-b840-d5911c52a7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:50.829912 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:50.848012 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-8248d79e-e4a6-4108-b840-d5911c52a7d4 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:36:50.849305 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-8248d79e-e4a6-4108-b840-d5911c52a7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:36:50.924439 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 620/1248] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:36:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:36:51.220403 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cfc023-da24-41c7-ba45-b09ade56d725 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:51.240111 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:51.333173 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cfc023-da24-41c7-ba45-b09ade56d725 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:51.371962 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e23b7-6868-4ae6-b9bd-d64b9a48ae62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:51.372285 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e23b7-6868-4ae6-b9bd-d64b9a48ae62 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:51.372626 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e23b7-6868-4ae6-b9bd-d64b9a48ae62 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:51.372970 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e23b7-6868-4ae6-b9bd-d64b9a48ae62 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:51.383763 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453e23b7-6868-4ae6-b9bd-d64b9a48ae62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:36:51.389589 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-453e23b7-6868-4ae6-b9bd-d64b9a48ae62 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:36:51.389589 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:36:51.389589 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:36:51.389589 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:36:51.389589 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:36:51.389589 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:36:51.389589 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:36:51.389589 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:36:51.389589 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:36:51.389589 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:36:51.389589 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:36:51.389589 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:36:51.389589 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:36:51.389589 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:36:51.389589 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:36:51.389589 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:36:51.390781 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:36:51.390781 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:36:51.390781 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:36:51.390781 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:36:51.390781 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:36:51.390781 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:36:51.390781 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:36:51.390781 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:36:51.390781 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:36:51.390781 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:36:51.390781 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:36:51.390781 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:36:51.390781 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:36:51.390781 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:36:51.390781 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:36:51.390781 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:36:51.392345 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:36:51.392345 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:36:51.392345 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:36:51.392345 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:36:51.392345 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:36:51.392345 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:36:51.392345 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:36:51.392345 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:36:51.392345 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:36:51.392345 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:36:51.392345 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:36:51.392345 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:36:51.392345 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 629/1249] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Sep 22 02:36:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:36:51.397381 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248d79e-e4a6-4108-b840-d5911c52a7d4 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:51.521356 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248d79e-e4a6-4108-b840-d5911c52a7d4 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:51.564992 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:51.572221 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70155cf4-7efa-415a-b434-210a83240c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:51.573680 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70155cf4-7efa-415a-b434-210a83240c69 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:51.574054 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70155cf4-7efa-415a-b434-210a83240c69 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:51.574398 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70155cf4-7efa-415a-b434-210a83240c69 None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:51.575242 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70155cf4-7efa-415a-b434-210a83240c69 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:36:51.575530 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70155cf4-7efa-415a-b434-210a83240c69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:51.577129 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70155cf4-7efa-415a-b434-210a83240c69 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:51.592783 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 621/1250] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:36:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:36:51.594149 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e23b7-6868-4ae6-b9bd-d64b9a48ae62 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:51.695296 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:51.734471 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e23b7-6868-4ae6-b9bd-d64b9a48ae62 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:51.775998 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb4bbef-d072-46c0-80cb-23aeb44c83f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:51.776752 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4bbef-d072-46c0-80cb-23aeb44c83f8 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:51.777074 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4bbef-d072-46c0-80cb-23aeb44c83f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:51.777411 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4bbef-d072-46c0-80cb-23aeb44c83f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:51.788880 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb4bbef-d072-46c0-80cb-23aeb44c83f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:51.790346 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb4bbef-d072-46c0-80cb-23aeb44c83f8 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:51.809689 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-ddb4bbef-d072-46c0-80cb-23aeb44c83f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:36:51.809689 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:36:51.809689 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:36:51.809689 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:36:51.809689 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:36:51.809689 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:36:51.809689 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:36:51.809689 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:36:51.809689 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:36:51.809689 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:36:51.809689 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:36:51.809689 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:36:51.809689 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:36:51.809689 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:36:51.809689 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:36:51.809689 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:36:51.810965 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:36:51.810965 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:36:51.810965 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:36:51.810965 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:36:51.810965 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:36:51.810965 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:36:51.810965 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:36:51.810965 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:36:51.810965 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:36:51.810965 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:36:51.810965 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:36:51.810965 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:36:51.810965 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:36:51.810965 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:36:51.810965 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:36:51.810965 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:36:51.810965 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:36:51.811950 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:36:51.811950 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:36:51.811950 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:36:51.815029 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 630/1251] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:36:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:36:51.817511 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70155cf4-7efa-415a-b434-210a83240c69 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:51.938106 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70155cf4-7efa-415a-b434-210a83240c69 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:51.977672 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39e32b-2041-4a68-87c8-81d369c4e8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:51.978507 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39e32b-2041-4a68-87c8-81d369c4e8ac None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:51.979356 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39e32b-2041-4a68-87c8-81d369c4e8ac None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:51.979638 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39e32b-2041-4a68-87c8-81d369c4e8ac None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:51.980301 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b39e32b-2041-4a68-87c8-81d369c4e8ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:36:51.980608 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b39e32b-2041-4a68-87c8-81d369c4e8ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:51.982074 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b39e32b-2041-4a68-87c8-81d369c4e8ac None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:51.996441 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 622/1252] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:36:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:36:52.002699 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb4bbef-d072-46c0-80cb-23aeb44c83f8 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:52.118545 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb4bbef-d072-46c0-80cb-23aeb44c83f8 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:52.150422 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:52.160609 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c3a84-200c-4511-ab1a-99497b9f8008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:52.161307 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c3a84-200c-4511-ab1a-99497b9f8008 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:52.161558 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c3a84-200c-4511-ab1a-99497b9f8008 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:52.161833 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c3a84-200c-4511-ab1a-99497b9f8008 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:52.170885 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255c3a84-200c-4511-ab1a-99497b9f8008 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:52.172409 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255c3a84-200c-4511-ab1a-99497b9f8008 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:52.188031 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-255c3a84-200c-4511-ab1a-99497b9f8008 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 22 02:36:52.188031 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:36:52.188031 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:36:52.188031 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:36:52.188031 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:36:52.188031 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:36:52.188031 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:36:52.188031 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:36:52.188031 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:36:52.188031 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:36:52.188031 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:36:52.188031 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:36:52.188031 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:36:52.188031 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:36:52.188031 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:36:52.188031 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:36:52.189147 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:36:52.189147 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:36:52.189147 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:36:52.189147 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:36:52.189147 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:36:52.189147 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:36:52.189147 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:36:52.189147 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:36:52.189147 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:36:52.189147 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:36:52.189147 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:36:52.189147 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:36:52.189147 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:36:52.189147 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:36:52.189147 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:36:52.189147 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:36:52.189147 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:36:52.189147 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:36:52.190932 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:36:52.190932 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:36:52.190932 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:36:52.190932 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:36:52.190932 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:36:52.190932 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:36:52.190932 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:36:52.190932 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:36:52.190932 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 22 02:36:52.190932 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:36:52.190932 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 631/1253] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:36:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:36:52.197256 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39e32b-2041-4a68-87c8-81d369c4e8ac None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:52.264377 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:52.316989 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39e32b-2041-4a68-87c8-81d369c4e8ac None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:52.375758 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccec4519-882f-4f9f-9347-564bea260b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:52.376311 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec4519-882f-4f9f-9347-564bea260b22 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:52.376595 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec4519-882f-4f9f-9347-564bea260b22 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:52.376872 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec4519-882f-4f9f-9347-564bea260b22 None admin] PATH_INFO: `/v3/users` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:52.377634 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccec4519-882f-4f9f-9347-564bea260b22 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:36:52.377941 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccec4519-882f-4f9f-9347-564bea260b22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:52.379471 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccec4519-882f-4f9f-9347-564bea260b22 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:52.399374 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 623/1254] 10.222.0.20 () {56 vars in 1143 bytes} [Thu Sep 22 02:36:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:36:52.405950 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c3a84-200c-4511-ab1a-99497b9f8008 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:52.550050 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c3a84-200c-4511-ab1a-99497b9f8008 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:52.599877 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d8c505-3579-4e54-bd33-808102795309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:52.600507 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8c505-3579-4e54-bd33-808102795309 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:52.600787 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8c505-3579-4e54-bd33-808102795309 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:52.601063 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8c505-3579-4e54-bd33-808102795309 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:52.671801 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:52.687390 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d8c505-3579-4e54-bd33-808102795309 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:52.688895 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d8c505-3579-4e54-bd33-808102795309 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:52.799655 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 632/1255] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:36:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 02:36:52.805379 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccec4519-882f-4f9f-9347-564bea260b22 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:52.931412 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccec4519-882f-4f9f-9347-564bea260b22 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:52.946199 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:52.969564 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0ce1c-7b71-4545-a346-f1fc89f00d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:52.970699 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0ce1c-7b71-4545-a346-f1fc89f00d58 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:52.971021 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0ce1c-7b71-4545-a346-f1fc89f00d58 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:52.971298 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0ce1c-7b71-4545-a346-f1fc89f00d58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:52.982528 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc0ce1c-7b71-4545-a346-f1fc89f00d58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:52.984053 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc0ce1c-7b71-4545-a346-f1fc89f00d58 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:52.999700 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-cbc0ce1c-7b71-4545-a346-f1fc89f00d58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:36:52.999700 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:36:52.999700 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:36:52.999700 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:36:52.999700 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:36:52.999700 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:36:52.999700 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:36:52.999700 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:36:52.999700 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:36:52.999700 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:36:52.999700 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:36:52.999700 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:36:52.999700 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:36:52.999700 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:36:52.999700 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:36:52.999700 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:36:53.000849 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:36:53.000849 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:36:53.000849 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:36:53.000849 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:36:53.000849 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:36:53.000849 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:36:53.000849 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:36:53.000849 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:36:53.000849 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:36:53.000849 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:36:53.000849 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:36:53.000849 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:36:53.000849 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:36:53.000849 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:36:53.000849 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:36:53.000849 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:36:53.000849 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:36:53.001915 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:36:53.001915 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:36:53.001915 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:36:53.001915 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 624/1256] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:36:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:36:53.006663 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d8c505-3579-4e54-bd33-808102795309 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:53.096069 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:53.109488 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d8c505-3579-4e54-bd33-808102795309 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:53.146148 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1285ceb-b142-469b-8147-6edf40db3aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:53.146757 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1285ceb-b142-469b-8147-6edf40db3aa0 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:53.147068 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1285ceb-b142-469b-8147-6edf40db3aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:53.147342 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1285ceb-b142-469b-8147-6edf40db3aa0 None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:53.147984 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1285ceb-b142-469b-8147-6edf40db3aa0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:36:53.148288 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1285ceb-b142-469b-8147-6edf40db3aa0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:53.149658 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1285ceb-b142-469b-8147-6edf40db3aa0 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:53.161658 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 633/1257] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:36:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:36:53.168365 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0ce1c-7b71-4545-a346-f1fc89f00d58 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:53.289254 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0ce1c-7b71-4545-a346-f1fc89f00d58 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:53.326227 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d3f77-ad80-4cc5-b414-30a4123a2524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:53.326898 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d3f77-ad80-4cc5-b414-30a4123a2524 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:53.327185 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d3f77-ad80-4cc5-b414-30a4123a2524 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:53.327468 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d3f77-ad80-4cc5-b414-30a4123a2524 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:53.334949 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51d3f77-ad80-4cc5-b414-30a4123a2524 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:53.336464 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51d3f77-ad80-4cc5-b414-30a4123a2524 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:53.349566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-e51d3f77-ad80-4cc5-b414-30a4123a2524 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:36:53.349566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:36:53.349566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:36:53.349566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:36:53.349566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:36:53.349566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:36:53.349566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:36:53.349566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:36:53.349566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:36:53.349566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:36:53.349566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:36:53.349566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:36:53.349566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:36:53.349566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:36:53.349566 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:36:53.350858 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:36:53.350858 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:36:53.350858 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:36:53.350858 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:36:53.350858 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:36:53.350858 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:36:53.350858 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:36:53.350858 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:36:53.350858 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:36:53.350858 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:36:53.350858 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:36:53.350858 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:36:53.350858 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:36:53.350858 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:36:53.350858 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:36:53.350858 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:36:53.350858 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:36:53.351956 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:36:53.351956 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:36:53.351956 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:36:53.351956 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:36:53.351956 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:36:53.351956 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:36:53.351956 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:36:53.351956 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:36:53.351956 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 625/1258] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:36:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:36:53.356974 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1285ceb-b142-469b-8147-6edf40db3aa0 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:53.459145 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1285ceb-b142-469b-8147-6edf40db3aa0 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:53.492020 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9aa3c-30b5-4e25-8a25-50d5fb98fba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:53.492628 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9aa3c-30b5-4e25-8a25-50d5fb98fba4 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:53.492910 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9aa3c-30b5-4e25-8a25-50d5fb98fba4 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:53.493186 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9aa3c-30b5-4e25-8a25-50d5fb98fba4 None admin] PATH_INFO: `/v3/projects` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:53.493876 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f9aa3c-30b5-4e25-8a25-50d5fb98fba4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:36:53.494182 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f9aa3c-30b5-4e25-8a25-50d5fb98fba4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:53.495682 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f9aa3c-30b5-4e25-8a25-50d5fb98fba4 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:53.508652 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 634/1259] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Sep 22 02:36:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:36:53.516567 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d3f77-ad80-4cc5-b414-30a4123a2524 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:53.549681 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:53.627151 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d3f77-ad80-4cc5-b414-30a4123a2524 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:53.664374 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92378cf-caa9-4877-b0aa-ba91a4f04b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:53.665080 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92378cf-caa9-4877-b0aa-ba91a4f04b7c None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:53.665368 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92378cf-caa9-4877-b0aa-ba91a4f04b7c None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:53.665748 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92378cf-caa9-4877-b0aa-ba91a4f04b7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:53.666664 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92378cf-caa9-4877-b0aa-ba91a4f04b7c None admin] RBAC: Adding query filter params (role.id=31a687548f55493b8ea666fe7e350303, scope.project.id=2a792f8dc45145d687ddd58c94d44b5e, user.id=865576e9e17d4efdbd1ca6a005f31bda) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:36:53.667058 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92378cf-caa9-4877-b0aa-ba91a4f04b7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:53.668850 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92378cf-caa9-4877-b0aa-ba91a4f04b7c None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:53.675283 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 626/1260] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:36:53 2022] GET /identity/v3/role_assignments?user.id=865576e9e17d4efdbd1ca6a005f31bda&scope.project.id=2a792f8dc45145d687ddd58c94d44b5e&role.id=31a687548f55493b8ea666fe7e350303 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:36:55.801990 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26080160-12a2-4abe-b015-0c9ab04be43a None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:55.802542 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26080160-12a2-4abe-b015-0c9ab04be43a None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:55.802542 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26080160-12a2-4abe-b015-0c9ab04be43a None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:55.803915 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 635/1261] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:36:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:36:55.809761 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200819f-b6fd-4fd3-9f0e-56e1d6977936 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:55.810150 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200819f-b6fd-4fd3-9f0e-56e1d6977936 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:55.810307 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200819f-b6fd-4fd3-9f0e-56e1d6977936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:55.828590 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-4200819f-b6fd-4fd3-9f0e-56e1d6977936 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:36:55.829776 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-4200819f-b6fd-4fd3-9f0e-56e1d6977936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:36:55.905710 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 627/1262] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:36:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:36:56.178180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26080160-12a2-4abe-b015-0c9ab04be43a None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:56.189205 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:56.288572 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26080160-12a2-4abe-b015-0c9ab04be43a None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:56.323126 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225696d8-c5fb-4b52-81ee-42eb7b4db58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:56.323758 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225696d8-c5fb-4b52-81ee-42eb7b4db58a None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:56.324044 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225696d8-c5fb-4b52-81ee-42eb7b4db58a None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:56.324321 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225696d8-c5fb-4b52-81ee-42eb7b4db58a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:56.334471 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225696d8-c5fb-4b52-81ee-42eb7b4db58a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:36:56.339989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-225696d8-c5fb-4b52-81ee-42eb7b4db58a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:36:56.339989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:36:56.339989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:36:56.339989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:36:56.339989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:36:56.339989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:36:56.339989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:36:56.339989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:36:56.339989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:36:56.339989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:36:56.339989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:36:56.339989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:36:56.339989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:36:56.339989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:36:56.339989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:36:56.339989 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:36:56.341235 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:36:56.341235 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:36:56.341235 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:36:56.341235 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:36:56.341235 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:36:56.341235 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:36:56.341235 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:36:56.341235 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:36:56.341235 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:36:56.341235 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:36:56.341235 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:36:56.341235 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:36:56.341235 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:36:56.341235 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:36:56.341235 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:36:56.341235 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:36:56.342369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:36:56.342369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:36:56.342369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:36:56.342369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:36:56.342369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:36:56.342369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:36:56.342369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:36:56.342369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:36:56.342369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:36:56.342369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:36:56.342369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:36:56.342369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:36:56.342369 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 636/1263] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Sep 22 02:36:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:36:56.347229 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200819f-b6fd-4fd3-9f0e-56e1d6977936 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:56.364880 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:56.443638 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200819f-b6fd-4fd3-9f0e-56e1d6977936 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:56.478052 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa4b4c-e3d2-42d3-b393-efd8f723f460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:56.478683 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa4b4c-e3d2-42d3-b393-efd8f723f460 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:56.478993 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa4b4c-e3d2-42d3-b393-efd8f723f460 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:56.479266 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa4b4c-e3d2-42d3-b393-efd8f723f460 None admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:56.479920 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baa4b4c-e3d2-42d3-b393-efd8f723f460 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:36:56.480227 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baa4b4c-e3d2-42d3-b393-efd8f723f460 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:56.481587 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baa4b4c-e3d2-42d3-b393-efd8f723f460 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:56.490083 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 628/1264] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:36:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:36:56.495914 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225696d8-c5fb-4b52-81ee-42eb7b4db58a None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:56.583638 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:56.620044 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225696d8-c5fb-4b52-81ee-42eb7b4db58a None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:56.654195 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de65f73-6233-4f4a-8156-b85dd8b9bec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:56.654847 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de65f73-6233-4f4a-8156-b85dd8b9bec2 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:56.655135 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de65f73-6233-4f4a-8156-b85dd8b9bec2 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:56.655408 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de65f73-6233-4f4a-8156-b85dd8b9bec2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:56.666206 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de65f73-6233-4f4a-8156-b85dd8b9bec2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:56.667685 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de65f73-6233-4f4a-8156-b85dd8b9bec2 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:56.682324 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-9de65f73-6233-4f4a-8156-b85dd8b9bec2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:36:56.682324 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:36:56.682324 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:36:56.682324 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:36:56.682324 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:36:56.682324 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:36:56.682324 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:36:56.682324 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:36:56.682324 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:36:56.682324 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:36:56.682324 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:36:56.682324 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:36:56.682324 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:36:56.682324 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:36:56.682324 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:36:56.682324 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:36:56.683711 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:36:56.683711 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:36:56.683711 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:36:56.683711 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:36:56.683711 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:36:56.683711 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:36:56.683711 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:36:56.683711 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:36:56.683711 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:36:56.683711 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:36:56.683711 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:36:56.683711 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:36:56.683711 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:36:56.683711 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:36:56.683711 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:36:56.683711 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:36:56.683711 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:36:56.684703 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:36:56.684703 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:36:56.684703 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:36:56.684703 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 637/1265] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:36:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:36:56.689293 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa4b4c-e3d2-42d3-b393-efd8f723f460 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:56.769622 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:56.789654 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa4b4c-e3d2-42d3-b393-efd8f723f460 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:56.823427 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05c96fd-457f-4447-b38c-f38f5d4bb730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:56.824096 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c96fd-457f-4447-b38c-f38f5d4bb730 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:56.824291 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c96fd-457f-4447-b38c-f38f5d4bb730 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:56.824559 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c96fd-457f-4447-b38c-f38f5d4bb730 None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:56.825195 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05c96fd-457f-4447-b38c-f38f5d4bb730 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:36:56.825495 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05c96fd-457f-4447-b38c-f38f5d4bb730 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:56.826864 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05c96fd-457f-4447-b38c-f38f5d4bb730 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:56.838406 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 629/1266] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:36:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:36:56.844168 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de65f73-6233-4f4a-8156-b85dd8b9bec2 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:56.951790 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de65f73-6233-4f4a-8156-b85dd8b9bec2 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:56.970761 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:56.985300 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f0dda-1aae-43dc-ad4b-893d5170846a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:56.985930 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f0dda-1aae-43dc-ad4b-893d5170846a None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:56.986209 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f0dda-1aae-43dc-ad4b-893d5170846a None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:56.986481 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f0dda-1aae-43dc-ad4b-893d5170846a None admin] PATH_INFO: `/v3/users/placement` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:56.993894 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9f0dda-1aae-43dc-ad4b-893d5170846a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:56.995931 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9f0dda-1aae-43dc-ad4b-893d5170846a None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:57.007093 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-fa9f0dda-1aae-43dc-ad4b-893d5170846a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 22 02:36:57.007093 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:36:57.007093 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:36:57.007093 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:36:57.007093 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:36:57.007093 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:36:57.007093 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:36:57.007093 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:36:57.007093 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:36:57.007093 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:36:57.007093 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:36:57.007093 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:36:57.007093 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:36:57.007093 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:36:57.007093 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:36:57.007093 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:36:57.008122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:36:57.008122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:36:57.008122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:36:57.008122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:36:57.008122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:36:57.008122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:36:57.008122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:36:57.008122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:36:57.008122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:36:57.008122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:36:57.008122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:36:57.008122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:36:57.008122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:36:57.008122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:36:57.008122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:36:57.008122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:36:57.008122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:36:57.008122 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:36:57.009692 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:36:57.009692 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:36:57.009692 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:36:57.009692 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:36:57.009692 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:36:57.009692 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:36:57.009692 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:36:57.009692 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:36:57.009692 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 22 02:36:57.009692 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:36:57.009692 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 638/1267] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:36:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:36:57.014161 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05c96fd-457f-4447-b38c-f38f5d4bb730 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:57.113691 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05c96fd-457f-4447-b38c-f38f5d4bb730 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:57.148357 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad92e72-f32a-47b7-ab02-0d6f72897130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:57.148967 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad92e72-f32a-47b7-ab02-0d6f72897130 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:57.149252 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad92e72-f32a-47b7-ab02-0d6f72897130 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:57.149524 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad92e72-f32a-47b7-ab02-0d6f72897130 None admin] PATH_INFO: `/v3/users` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:57.150255 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad92e72-f32a-47b7-ab02-0d6f72897130 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:36:57.150596 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad92e72-f32a-47b7-ab02-0d6f72897130 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:57.152086 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad92e72-f32a-47b7-ab02-0d6f72897130 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:57.160974 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:57.170527 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 630/1268] 10.222.0.20 () {56 vars in 1143 bytes} [Thu Sep 22 02:36:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 02:36:57.176390 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f0dda-1aae-43dc-ad4b-893d5170846a None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:57.277253 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f0dda-1aae-43dc-ad4b-893d5170846a None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:57.312831 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c5f17b-ae09-43cd-979f-c9f5dbce8719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:57.313543 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5f17b-ae09-43cd-979f-c9f5dbce8719 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:57.313821 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5f17b-ae09-43cd-979f-c9f5dbce8719 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:57.314248 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5f17b-ae09-43cd-979f-c9f5dbce8719 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:57.324862 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c5f17b-ae09-43cd-979f-c9f5dbce8719 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:57.326298 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c5f17b-ae09-43cd-979f-c9f5dbce8719 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:57.340272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-35c5f17b-ae09-43cd-979f-c9f5dbce8719 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:36:57.340272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:36:57.340272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:36:57.340272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:36:57.340272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:36:57.340272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:36:57.340272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:36:57.340272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:36:57.340272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:36:57.340272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:36:57.340272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:36:57.340272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:36:57.340272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:36:57.340272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:36:57.340272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:36:57.340272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:36:57.341471 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:36:57.341471 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:36:57.341471 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:36:57.341471 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:36:57.341471 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:36:57.341471 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:36:57.341471 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:36:57.341471 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:36:57.341471 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:36:57.341471 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:36:57.341471 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:36:57.341471 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:36:57.341471 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:36:57.341471 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:36:57.341471 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:36:57.341471 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:36:57.341471 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:36:57.342665 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:36:57.342665 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:36:57.342665 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:36:57.342665 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 639/1269] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:36:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 02:36:57.347122 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad92e72-f32a-47b7-ab02-0d6f72897130 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:57.445603 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad92e72-f32a-47b7-ab02-0d6f72897130 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:57.479644 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3debfc47-c967-43a1-acc6-3e5e15ee4184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:57.480244 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debfc47-c967-43a1-acc6-3e5e15ee4184 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:57.480525 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debfc47-c967-43a1-acc6-3e5e15ee4184 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:57.480797 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debfc47-c967-43a1-acc6-3e5e15ee4184 None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:57.481434 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3debfc47-c967-43a1-acc6-3e5e15ee4184 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:36:57.481738 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3debfc47-c967-43a1-acc6-3e5e15ee4184 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:57.483120 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3debfc47-c967-43a1-acc6-3e5e15ee4184 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:57.494163 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 631/1270] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:36:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:36:57.500064 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c5f17b-ae09-43cd-979f-c9f5dbce8719 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:57.543577 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:57.616806 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c5f17b-ae09-43cd-979f-c9f5dbce8719 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:57.651622 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01801f3-c4fe-4e02-b89c-be6d5436fcbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:57.652308 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01801f3-c4fe-4e02-b89c-be6d5436fcbc None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:57.652694 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01801f3-c4fe-4e02-b89c-be6d5436fcbc None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:57.653019 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01801f3-c4fe-4e02-b89c-be6d5436fcbc None admin] PATH_INFO: `/v3/projects/service` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:57.660843 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01801f3-c4fe-4e02-b89c-be6d5436fcbc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:57.662299 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01801f3-c4fe-4e02-b89c-be6d5436fcbc None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:57.674289 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-a01801f3-c4fe-4e02-b89c-be6d5436fcbc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:36:57.674289 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:36:57.674289 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:36:57.674289 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:36:57.674289 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:36:57.674289 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:36:57.674289 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:36:57.674289 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:36:57.674289 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:36:57.674289 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:36:57.674289 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:36:57.674289 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:36:57.674289 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:36:57.674289 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:36:57.674289 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:36:57.675465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:36:57.675465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:36:57.675465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:36:57.675465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:36:57.675465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:36:57.675465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:36:57.675465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:36:57.675465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:36:57.675465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:36:57.675465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:36:57.675465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:36:57.675465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:36:57.675465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:36:57.675465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:36:57.675465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:36:57.675465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:36:57.675465 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:36:57.676675 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:36:57.676675 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:36:57.676675 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:36:57.676675 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:36:57.676675 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:36:57.676675 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:36:57.676675 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:36:57.676675 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:36:57.676675 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 640/1271] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:36:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:36:57.681273 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3debfc47-c967-43a1-acc6-3e5e15ee4184 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:57.740157 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:57.783855 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3debfc47-c967-43a1-acc6-3e5e15ee4184 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:57.816047 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5710959f-3a3c-4eed-9a66-208e17db8314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:57.816579 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5710959f-3a3c-4eed-9a66-208e17db8314 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:57.816858 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5710959f-3a3c-4eed-9a66-208e17db8314 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:57.817133 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5710959f-3a3c-4eed-9a66-208e17db8314 None admin] PATH_INFO: `/v3/projects` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:57.817822 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5710959f-3a3c-4eed-9a66-208e17db8314 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:36:57.818132 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5710959f-3a3c-4eed-9a66-208e17db8314 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:57.819591 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5710959f-3a3c-4eed-9a66-208e17db8314 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:57.831959 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 632/1272] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Sep 22 02:36:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:36:57.839574 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01801f3-c4fe-4e02-b89c-be6d5436fcbc None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:36:57.936495 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:36:57.946225 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01801f3-c4fe-4e02-b89c-be6d5436fcbc None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:36:57.982858 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0386b2b-1e0b-4002-94f2-ea843605e725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:36:57.983547 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0386b2b-1e0b-4002-94f2-ea843605e725 None admin] REQUEST_METHOD: `PUT` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:36:57.983831 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0386b2b-1e0b-4002-94f2-ea843605e725 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:36:57.984109 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0386b2b-1e0b-4002-94f2-ea843605e725 None admin] PATH_INFO: `/v3/projects/2a792f8dc45145d687ddd58c94d44b5e/users/865576e9e17d4efdbd1ca6a005f31bda/roles/31a687548f55493b8ea666fe7e350303` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:36:57.986057 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0386b2b-1e0b-4002-94f2-ea843605e725 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a792f8dc45145d687ddd58c94d44b5e, user_id=865576e9e17d4efdbd1ca6a005f31bda, role_id=31a687548f55493b8ea666fe7e350303)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:36:57.987547 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0386b2b-1e0b-4002-94f2-ea843605e725 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:36:58.001436 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 641/1273] 10.222.0.20 () {58 vars in 1439 bytes} [Thu Sep 22 02:36:57 2022] PUT /identity/v3/projects/2a792f8dc45145d687ddd58c94d44b5e/users/865576e9e17d4efdbd1ca6a005f31bda/roles/31a687548f55493b8ea666fe7e350303 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 02:37:00.171082 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5f0d2-4b37-4e6e-9260-5e81e59721c1 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:37:00.171631 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5f0d2-4b37-4e6e-9260-5e81e59721c1 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:37:00.171631 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5f0d2-4b37-4e6e-9260-5e81e59721c1 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:37:00.172971 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 633/1274] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:37:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:37:00.178171 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b9a13-c161-47f7-b39f-124c02eb938c None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:37:00.178604 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b9a13-c161-47f7-b39f-124c02eb938c None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:37:00.178736 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b9a13-c161-47f7-b39f-124c02eb938c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:37:00.196742 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-934b9a13-c161-47f7-b39f-124c02eb938c None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:37:00.198454 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-934b9a13-c161-47f7-b39f-124c02eb938c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:37:00.266255 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 642/1275] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:37:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:37:00.530060 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5f0d2-4b37-4e6e-9260-5e81e59721c1 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:37:00.636758 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5f0d2-4b37-4e6e-9260-5e81e59721c1 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:37:00.655053 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:37:00.672764 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b746754-9e5d-41a8-b19b-7c18af159209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:37:00.673400 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b746754-9e5d-41a8-b19b-7c18af159209 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:37:00.673685 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b746754-9e5d-41a8-b19b-7c18af159209 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:37:00.673961 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b746754-9e5d-41a8-b19b-7c18af159209 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:37:00.684608 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b746754-9e5d-41a8-b19b-7c18af159209 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 02:37:00.689491 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-6b746754-9e5d-41a8-b19b-7c18af159209 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:37:00.689491 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:37:00.689491 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:37:00.689491 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:37:00.689491 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:37:00.689491 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:37:00.689491 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:37:00.689491 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:37:00.689491 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:37:00.689491 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:37:00.689491 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:37:00.689491 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:37:00.689491 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 02:37:00.689491 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 02:37:00.689491 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 02:37:00.689491 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 02:37:00.690623 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 02:37:00.690623 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 02:37:00.690623 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 02:37:00.690623 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 02:37:00.690623 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:37:00.690623 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:37:00.690623 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:37:00.690623 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:37:00.690623 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:37:00.690623 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:37:00.690623 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:37:00.690623 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:37:00.690623 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:37:00.690623 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:37:00.690623 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:37:00.690623 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:37:00.691549 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:37:00.691549 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:37:00.691549 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:37:00.691549 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:37:00.691549 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 02:37:00.691549 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 02:37:00.691549 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 02:37:00.691549 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 02:37:00.691549 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 02:37:00.691549 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 02:37:00.691549 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 02:37:00.691549 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:37:00.691549 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 634/1276] 10.222.0.20 () {56 vars in 1096 bytes} [Thu Sep 22 02:37:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:37:00.696826 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934b9a13-c161-47f7-b39f-124c02eb938c None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:37:00.778436 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:37:00.797263 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934b9a13-c161-47f7-b39f-124c02eb938c None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:37:00.830851 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd66186-a7af-4cb7-aef9-510f7901cfba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:37:00.831460 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd66186-a7af-4cb7-aef9-510f7901cfba None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:37:00.831744 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd66186-a7af-4cb7-aef9-510f7901cfba None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:37:00.832054 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd66186-a7af-4cb7-aef9-510f7901cfba None admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:37:00.832691 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd66186-a7af-4cb7-aef9-510f7901cfba None admin] RBAC: Adding query filter params (name=admin) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:37:00.832993 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd66186-a7af-4cb7-aef9-510f7901cfba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:37:00.834360 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd66186-a7af-4cb7-aef9-510f7901cfba None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:37:00.842508 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 643/1277] 10.222.0.20 () {56 vars in 1099 bytes} [Thu Sep 22 02:37:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:37:00.848522 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b746754-9e5d-41a8-b19b-7c18af159209 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:37:00.960195 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b746754-9e5d-41a8-b19b-7c18af159209 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:37:00.997077 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af88de-4c61-4818-81ac-65accfad8520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:37:00.997732 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af88de-4c61-4818-81ac-65accfad8520 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:37:00.998018 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af88de-4c61-4818-81ac-65accfad8520 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:37:00.998323 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af88de-4c61-4818-81ac-65accfad8520 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:37:01.009300 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2af88de-4c61-4818-81ac-65accfad8520 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:37:01.010795 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2af88de-4c61-4818-81ac-65accfad8520 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:37:01.027217 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-d2af88de-4c61-4818-81ac-65accfad8520 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:37:01.027217 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:37:01.027217 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:37:01.027217 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:37:01.027217 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:37:01.027217 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:37:01.027217 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:37:01.027217 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:37:01.027217 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:37:01.027217 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:37:01.027217 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:37:01.027217 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:37:01.027217 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:37:01.027217 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:37:01.027217 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:37:01.027217 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:37:01.028375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:37:01.028375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:37:01.028375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:37:01.028375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:37:01.028375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:37:01.028375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:37:01.028375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:37:01.028375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:37:01.028375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:37:01.028375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:37:01.028375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:37:01.028375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:37:01.028375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:37:01.028375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:37:01.028375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:37:01.028375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:37:01.028375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:37:01.034409 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:37:01.034409 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:37:01.034409 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:37:01.034409 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 635/1278] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:37:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:37:01.034710 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd66186-a7af-4cb7-aef9-510f7901cfba None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:37:01.135061 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd66186-a7af-4cb7-aef9-510f7901cfba None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:37:01.170439 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43edb16b-9e6f-4d0a-a68b-f53c43607d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:37:01.171079 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43edb16b-9e6f-4d0a-a68b-f53c43607d9b None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:37:01.171368 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43edb16b-9e6f-4d0a-a68b-f53c43607d9b None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:37:01.171643 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43edb16b-9e6f-4d0a-a68b-f53c43607d9b None admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:37:01.172286 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43edb16b-9e6f-4d0a-a68b-f53c43607d9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:37:01.172593 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43edb16b-9e6f-4d0a-a68b-f53c43607d9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:37:01.173958 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43edb16b-9e6f-4d0a-a68b-f53c43607d9b None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:37:01.185743 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 644/1279] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:37:01.191447 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af88de-4c61-4818-81ac-65accfad8520 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:37:01.222038 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:37:01.292338 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af88de-4c61-4818-81ac-65accfad8520 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:37:01.325633 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a7367-182a-49e8-8dc9-a8478ff1befb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:37:01.326269 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a7367-182a-49e8-8dc9-a8478ff1befb None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:37:01.326553 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a7367-182a-49e8-8dc9-a8478ff1befb None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:37:01.326845 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a7367-182a-49e8-8dc9-a8478ff1befb None admin] PATH_INFO: `/v3/users/placement` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:37:01.334219 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87a7367-182a-49e8-8dc9-a8478ff1befb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:37:01.335686 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87a7367-182a-49e8-8dc9-a8478ff1befb None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:37:01.347461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-f87a7367-182a-49e8-8dc9-a8478ff1befb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 22 02:37:01.347461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:37:01.347461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:37:01.347461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:37:01.347461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:37:01.347461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:37:01.347461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:37:01.347461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:37:01.347461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:37:01.347461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:37:01.347461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:37:01.347461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:37:01.347461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 02:37:01.347461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 02:37:01.347461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 02:37:01.347461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 02:37:01.348605 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:37:01.348605 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:37:01.348605 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 02:37:01.348605 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:37:01.348605 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 02:37:01.348605 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 02:37:01.348605 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:37:01.348605 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:37:01.348605 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:37:01.348605 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:37:01.348605 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:37:01.348605 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:37:01.348605 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:37:01.348605 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:37:01.348605 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:37:01.348605 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:37:01.348605 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:37:01.348605 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:37:01.349593 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:37:01.349593 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:37:01.349593 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 02:37:01.349593 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 02:37:01.349593 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 02:37:01.349593 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 02:37:01.349593 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 02:37:01.349593 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 02:37:01.349593 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 22 02:37:01.349593 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:37:01.349593 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 636/1280] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:37:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:37:01.354808 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43edb16b-9e6f-4d0a-a68b-f53c43607d9b None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:37:01.358615 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:37:01.453382 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43edb16b-9e6f-4d0a-a68b-f53c43607d9b None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:37:01.487960 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf4ef91-0ec8-4c1d-8ab2-82111ffa17e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:37:01.488566 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf4ef91-0ec8-4c1d-8ab2-82111ffa17e0 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:37:01.488851 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf4ef91-0ec8-4c1d-8ab2-82111ffa17e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:37:01.489128 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf4ef91-0ec8-4c1d-8ab2-82111ffa17e0 None admin] PATH_INFO: `/v3/users` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:37:01.489855 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf4ef91-0ec8-4c1d-8ab2-82111ffa17e0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:37:01.490167 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf4ef91-0ec8-4c1d-8ab2-82111ffa17e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:37:01.491624 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf4ef91-0ec8-4c1d-8ab2-82111ffa17e0 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:37:01.510500 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 645/1281] 10.222.0.20 () {56 vars in 1143 bytes} [Thu Sep 22 02:37:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:37:01.517059 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a7367-182a-49e8-8dc9-a8478ff1befb None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:37:01.608180 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:37:01.618143 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a7367-182a-49e8-8dc9-a8478ff1befb None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:37:01.653569 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb5a634-243e-4b8d-82db-31865b0f68ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:37:01.654223 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5a634-243e-4b8d-82db-31865b0f68ce None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:37:01.654596 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5a634-243e-4b8d-82db-31865b0f68ce None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:37:01.655009 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5a634-243e-4b8d-82db-31865b0f68ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:37:01.721075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb5a634-243e-4b8d-82db-31865b0f68ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:37:01.722439 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb5a634-243e-4b8d-82db-31865b0f68ce None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:37:01.809917 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:37:01.820359 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 637/1282] 10.222.0.20 () {58 vars in 1282 bytes} [Thu Sep 22 02:37:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 02:37:01.826781 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf4ef91-0ec8-4c1d-8ab2-82111ffa17e0 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:37:01.888041 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:37:01.928360 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf4ef91-0ec8-4c1d-8ab2-82111ffa17e0 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:37:01.962661 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c1b100-8333-4c39-a932-89758d78a859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:37:01.963314 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1b100-8333-4c39-a932-89758d78a859 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:37:01.963594 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1b100-8333-4c39-a932-89758d78a859 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:37:01.963865 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1b100-8333-4c39-a932-89758d78a859 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:37:01.973685 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c1b100-8333-4c39-a932-89758d78a859 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:37:01.975119 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c1b100-8333-4c39-a932-89758d78a859 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:37:01.989158 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-50c1b100-8333-4c39-a932-89758d78a859 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:37:01.989158 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:37:01.989158 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:37:01.989158 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:37:01.989158 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:37:01.989158 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:37:01.989158 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:37:01.989158 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:37:01.989158 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:37:01.989158 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:37:01.989158 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:37:01.989158 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:37:01.989158 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 02:37:01.989158 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 02:37:01.989158 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 02:37:01.989158 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 02:37:01.990291 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:37:01.990291 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:37:01.990291 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:37:01.990291 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:37:01.990291 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:37:01.990291 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:37:01.990291 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:37:01.990291 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:37:01.990291 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:37:01.990291 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:37:01.990291 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:37:01.990291 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:37:01.990291 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:37:01.990291 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:37:01.990291 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:37:01.990291 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:37:01.990291 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 02:37:01.991630 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 02:37:01.991630 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 02:37:01.991630 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:37:01.991630 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 646/1283] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:37:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:37:01.996480 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb5a634-243e-4b8d-82db-31865b0f68ce None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:37:02.093405 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb5a634-243e-4b8d-82db-31865b0f68ce None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:37:02.126883 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f9690c-f27d-46e8-b4da-5594f2cfd705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:37:02.127501 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9690c-f27d-46e8-b4da-5594f2cfd705 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:37:02.127787 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9690c-f27d-46e8-b4da-5594f2cfd705 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:37:02.128065 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9690c-f27d-46e8-b4da-5594f2cfd705 None admin] PATH_INFO: `/v3/domains` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:37:02.128715 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f9690c-f27d-46e8-b4da-5594f2cfd705 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:37:02.129023 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f9690c-f27d-46e8-b4da-5594f2cfd705 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:37:02.130387 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f9690c-f27d-46e8-b4da-5594f2cfd705 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:37:02.142170 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 638/1284] 10.222.0.20 () {56 vars in 1109 bytes} [Thu Sep 22 02:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:37:02.148099 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c1b100-8333-4c39-a932-89758d78a859 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:37:02.245080 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c1b100-8333-4c39-a932-89758d78a859 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:37:02.261916 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:37:02.281653 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deef0e6-98cc-4ca4-9419-f81781455846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:37:02.282277 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deef0e6-98cc-4ca4-9419-f81781455846 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:37:02.282555 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deef0e6-98cc-4ca4-9419-f81781455846 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:37:02.282852 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deef0e6-98cc-4ca4-9419-f81781455846 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:37:02.289215 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deef0e6-98cc-4ca4-9419-f81781455846 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:37:02.291165 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deef0e6-98cc-4ca4-9419-f81781455846 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:37:02.302074 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application [None req-1deef0e6-98cc-4ca4-9419-f81781455846 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:37:02.302074 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:37:02.302074 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:37:02.302074 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:37:02.302074 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:37:02.302074 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:37:02.302074 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:37:02.302074 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:37:02.302074 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:37:02.302074 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:37:02.302074 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:37:02.302074 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:37:02.302074 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 02:37:02.302074 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 02:37:02.302074 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 02:37:02.303189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 02:37:02.303189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:37:02.303189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:37:02.303189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:37:02.303189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:37:02.303189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:37:02.303189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:37:02.303189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:37:02.303189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application with Lock( Sep 22 02:37:02.303189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:37:02.303189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:37:02.303189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:37:02.303189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:37:02.303189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:37:02.303189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:37:02.303189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:37:02.303189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:37:02.304379 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 02:37:02.304379 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 02:37:02.304379 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 02:37:02.304379 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 02:37:02.304379 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 02:37:02.304379 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 02:37:02.304379 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 02:37:02.304379 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: ERROR keystone.server.flask.application Sep 22 02:37:02.304379 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 647/1285] 10.222.0.20 () {56 vars in 1111 bytes} [Thu Sep 22 02:37:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:37:02.309877 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f9690c-f27d-46e8-b4da-5594f2cfd705 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:37:02.358054 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:37:02.406019 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f9690c-f27d-46e8-b4da-5594f2cfd705 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:37:02.438956 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ad35b-30f2-4c97-be05-62a6d060607f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:37:02.439555 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ad35b-30f2-4c97-be05-62a6d060607f None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:37:02.439837 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ad35b-30f2-4c97-be05-62a6d060607f None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:37:02.440110 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ad35b-30f2-4c97-be05-62a6d060607f None admin] PATH_INFO: `/v3/projects` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:37:02.440795 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140ad35b-30f2-4c97-be05-62a6d060607f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:37:02.441104 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140ad35b-30f2-4c97-be05-62a6d060607f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:37:02.442514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140ad35b-30f2-4c97-be05-62a6d060607f None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:37:02.454108 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 639/1286] 10.222.0.20 () {56 vars in 1148 bytes} [Thu Sep 22 02:37:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:37:02.460205 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deef0e6-98cc-4ca4-9419-f81781455846 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:37:02.560258 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deef0e6-98cc-4ca4-9419-f81781455846 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:37:02.592148 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e25159-2ce8-4041-ba09-1a3b632bba37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:37:02.592751 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e25159-2ce8-4041-ba09-1a3b632bba37 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:37:02.593031 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e25159-2ce8-4041-ba09-1a3b632bba37 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:37:02.593304 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e25159-2ce8-4041-ba09-1a3b632bba37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:37:02.594025 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e25159-2ce8-4041-ba09-1a3b632bba37 None admin] RBAC: Adding query filter params (role.id=31a687548f55493b8ea666fe7e350303, scope.project.id=2a792f8dc45145d687ddd58c94d44b5e, user.id=865576e9e17d4efdbd1ca6a005f31bda) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:37:02.594336 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e25159-2ce8-4041-ba09-1a3b632bba37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:37:02.595795 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e25159-2ce8-4041-ba09-1a3b632bba37 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:37:02.601890 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 648/1287] 10.222.0.20 () {56 vars in 1374 bytes} [Thu Sep 22 02:37:02 2022] GET /identity/v3/role_assignments?user.id=865576e9e17d4efdbd1ca6a005f31bda&scope.project.id=2a792f8dc45145d687ddd58c94d44b5e&role.id=31a687548f55493b8ea666fe7e350303 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:37:04.774496 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2565776-9dcf-4fd1-8e31-328ca42487af None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:37:04.775107 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2565776-9dcf-4fd1-8e31-328ca42487af None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:37:04.775107 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2565776-9dcf-4fd1-8e31-328ca42487af None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:37:04.776975 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 640/1288] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:37:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:37:04.782105 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d7b6d-5395-42c7-8420-8ecb0aad9503 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:37:04.782533 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d7b6d-5395-42c7-8420-8ecb0aad9503 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:37:04.782684 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d7b6d-5395-42c7-8420-8ecb0aad9503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:37:04.800777 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-0d5d7b6d-5395-42c7-8420-8ecb0aad9503 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:37:04.801973 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-0d5d7b6d-5395-42c7-8420-8ecb0aad9503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:37:04.831201 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:37:04.868580 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 649/1289] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:37:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:37:05.148642 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2565776-9dcf-4fd1-8e31-328ca42487af None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:37:05.248563 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2565776-9dcf-4fd1-8e31-328ca42487af None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:37:05.259040 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:37:05.282111 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01143577-44ad-4dab-9dea-2d787344eb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:37:05.282748 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01143577-44ad-4dab-9dea-2d787344eb30 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:37:05.283062 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01143577-44ad-4dab-9dea-2d787344eb30 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:37:05.283339 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01143577-44ad-4dab-9dea-2d787344eb30 None admin] PATH_INFO: `/v3/services/placement` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:37:05.283921 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01143577-44ad-4dab-9dea-2d787344eb30 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:37:05.285307 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01143577-44ad-4dab-9dea-2d787344eb30 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:37:05.294788 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-01143577-44ad-4dab-9dea-2d787344eb30 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 02:37:05.294788 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:37:05.294788 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 02:37:05.294788 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 02:37:05.294788 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 02:37:05.294788 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 02:37:05.294788 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 02:37:05.294788 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:37:05.294788 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 02:37:05.294788 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:37:05.294788 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 02:37:05.294788 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:37:05.294788 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:37:05.294788 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:37:05.294788 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:37:05.294788 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:37:05.294788 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:37:05.296132 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:37:05.296132 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:37:05.296132 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:37:05.296132 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:37:05.296132 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:37:05.296132 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:37:05.296132 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 02:37:05.296132 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 02:37:05.296132 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 02:37:05.296132 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 02:37:05.296132 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:37:05.296132 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:37:05.296132 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:37:05.296132 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:37:05.296132 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:37:05.296132 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:37:05.297208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:37:05.297208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:37:05.297208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:37:05.297208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:37:05.297208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:37:05.297208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:37:05.297208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:37:05.297208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:37:05.297208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:37:05.297208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:37:05.297208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 02:37:05.297208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:37:05.297208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 02:37:05.297208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:37:05.297208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 641/1290] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Sep 22 02:37:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:37:05.301966 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d7b6d-5395-42c7-8420-8ecb0aad9503 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:37:05.408438 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d7b6d-5395-42c7-8420-8ecb0aad9503 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:37:05.444180 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff02ac11-2181-48e5-b689-308a7b7aa38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:37:05.444794 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff02ac11-2181-48e5-b689-308a7b7aa38a None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:37:05.445080 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff02ac11-2181-48e5-b689-308a7b7aa38a None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:37:05.445359 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff02ac11-2181-48e5-b689-308a7b7aa38a None admin] PATH_INFO: `/v3/services` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:37:05.445993 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff02ac11-2181-48e5-b689-308a7b7aa38a None admin] RBAC: Adding query filter params (name=placement) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:37:05.446329 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff02ac11-2181-48e5-b689-308a7b7aa38a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:37:05.447798 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff02ac11-2181-48e5-b689-308a7b7aa38a None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:37:05.452676 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 650/1291] 10.222.0.20 () {56 vars in 1116 bytes} [Thu Sep 22 02:37:05 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:37:05.458496 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01143577-44ad-4dab-9dea-2d787344eb30 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:37:05.562245 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01143577-44ad-4dab-9dea-2d787344eb30 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:37:05.595381 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c689c9-6eca-4243-b42f-f0b7584ff567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:37:05.595992 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c689c9-6eca-4243-b42f-f0b7584ff567 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:37:05.596277 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c689c9-6eca-4243-b42f-f0b7584ff567 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:37:05.596554 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c689c9-6eca-4243-b42f-f0b7584ff567 None admin] PATH_INFO: `/v3/services` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:37:05.597187 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c689c9-6eca-4243-b42f-f0b7584ff567 None admin] RBAC: Adding query filter params (type=placement) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:37:05.597492 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c689c9-6eca-4243-b42f-f0b7584ff567 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:37:05.598906 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c689c9-6eca-4243-b42f-f0b7584ff567 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:37:05.603981 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 642/1292] 10.222.0.20 () {56 vars in 1116 bytes} [Thu Sep 22 02:37:05 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:37:07.806225 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbcdc53-f893-4352-8510-2c0ebc7b7dfc None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:37:07.806761 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbcdc53-f893-4352-8510-2c0ebc7b7dfc None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:37:07.806761 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbcdc53-f893-4352-8510-2c0ebc7b7dfc None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:37:07.808155 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 651/1293] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:37:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:37:07.814031 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c45d5-5ebd-4765-ae90-274b41f8c1e7 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:37:07.814462 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c45d5-5ebd-4765-ae90-274b41f8c1e7 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:37:07.814587 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c45d5-5ebd-4765-ae90-274b41f8c1e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:37:07.832546 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-d43c45d5-5ebd-4765-ae90-274b41f8c1e7 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:37:07.833726 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-d43c45d5-5ebd-4765-ae90-274b41f8c1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:37:07.862285 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:37:07.899516 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 643/1294] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:37:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:37:08.168953 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbcdc53-f893-4352-8510-2c0ebc7b7dfc None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:37:08.199107 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:37:08.267991 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbcdc53-f893-4352-8510-2c0ebc7b7dfc None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:37:08.300937 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb67b75-c9d6-4099-86be-51b647d91bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:37:08.301536 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb67b75-c9d6-4099-86be-51b647d91bfc None admin] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:37:08.301819 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb67b75-c9d6-4099-86be-51b647d91bfc None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:37:08.302094 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb67b75-c9d6-4099-86be-51b647d91bfc None admin] PATH_INFO: `/v3/services` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:37:08.302742 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb67b75-c9d6-4099-86be-51b647d91bfc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:37:08.304136 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb67b75-c9d6-4099-86be-51b647d91bfc None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:37:08.319136 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 652/1295] 10.222.0.20 () {60 vars in 1141 bytes} [Thu Sep 22 02:37:08 2022] POST /identity/v3/services => generated 246 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:37:10.467672 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703bdd2-b02b-4445-9692-8b69984e1d68 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:37:10.468243 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703bdd2-b02b-4445-9692-8b69984e1d68 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:37:10.468243 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703bdd2-b02b-4445-9692-8b69984e1d68 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:37:10.469585 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 644/1296] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:37:10.475142 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0b292-bea3-4e94-8bd1-e268e8225ed5 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:37:10.475585 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0b292-bea3-4e94-8bd1-e268e8225ed5 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:37:10.475698 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0b292-bea3-4e94-8bd1-e268e8225ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:37:10.495631 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-87b0b292-bea3-4e94-8bd1-e268e8225ed5 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:37:10.496846 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-87b0b292-bea3-4e94-8bd1-e268e8225ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:37:10.588808 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 653/1297] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:37:10 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:37:10.855254 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9703bdd2-b02b-4445-9692-8b69984e1d68 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:37:10.970293 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9703bdd2-b02b-4445-9692-8b69984e1d68 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:37:11.005778 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a159f-f5ba-4a99-9f2b-4d05e9119eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:37:11.006421 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a159f-f5ba-4a99-9f2b-4d05e9119eef None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:37:11.006706 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a159f-f5ba-4a99-9f2b-4d05e9119eef None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:37:11.007062 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a159f-f5ba-4a99-9f2b-4d05e9119eef None admin] PATH_INFO: `/v3/services/placement` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:37:11.007756 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a159f-f5ba-4a99-9f2b-4d05e9119eef None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:37:11.009246 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a159f-f5ba-4a99-9f2b-4d05e9119eef None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:37:11.018765 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-5a7a159f-f5ba-4a99-9f2b-4d05e9119eef None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 02:37:11.018765 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:37:11.018765 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 02:37:11.018765 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 02:37:11.018765 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 02:37:11.018765 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 02:37:11.018765 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 02:37:11.018765 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:37:11.018765 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 02:37:11.018765 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:37:11.018765 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 02:37:11.018765 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:37:11.018765 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:37:11.018765 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:37:11.018765 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:37:11.018765 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:37:11.018765 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:37:11.019891 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:37:11.019891 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:37:11.019891 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:37:11.019891 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:37:11.019891 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:37:11.019891 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:37:11.019891 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 02:37:11.019891 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 02:37:11.019891 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 02:37:11.019891 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 02:37:11.019891 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:37:11.019891 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:37:11.019891 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:37:11.019891 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:37:11.019891 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:37:11.019891 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:37:11.020802 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:37:11.020802 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:37:11.020802 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:37:11.020802 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:37:11.020802 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:37:11.020802 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:37:11.020802 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:37:11.020802 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:37:11.020802 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:37:11.020802 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:37:11.020802 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 02:37:11.020802 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:37:11.020802 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 02:37:11.020802 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:37:11.020802 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 645/1298] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Sep 22 02:37:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:37:11.026553 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b0b292-bea3-4e94-8bd1-e268e8225ed5 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:37:11.045694 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:37:11.129233 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b0b292-bea3-4e94-8bd1-e268e8225ed5 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:37:11.164435 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e30fefe-073a-4641-a476-51f0902fb569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:37:11.165054 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e30fefe-073a-4641-a476-51f0902fb569 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:37:11.165892 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e30fefe-073a-4641-a476-51f0902fb569 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:37:11.166170 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e30fefe-073a-4641-a476-51f0902fb569 None admin] PATH_INFO: `/v3/services` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:37:11.166800 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e30fefe-073a-4641-a476-51f0902fb569 None admin] RBAC: Adding query filter params (name=placement) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:37:11.167138 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e30fefe-073a-4641-a476-51f0902fb569 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:37:11.168510 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e30fefe-073a-4641-a476-51f0902fb569 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:37:11.173687 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 654/1299] 10.222.0.20 () {56 vars in 1116 bytes} [Thu Sep 22 02:37:11 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:37:11.179889 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a159f-f5ba-4a99-9f2b-4d05e9119eef None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:37:11.211136 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:37:11.284053 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a159f-f5ba-4a99-9f2b-4d05e9119eef None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:37:11.317609 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450c3e1d-c8a1-4136-a259-e6cd91f429bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:37:11.318225 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c3e1d-c8a1-4136-a259-e6cd91f429bc None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:37:11.318512 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c3e1d-c8a1-4136-a259-e6cd91f429bc None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:37:11.318790 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c3e1d-c8a1-4136-a259-e6cd91f429bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:37:11.319527 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450c3e1d-c8a1-4136-a259-e6cd91f429bc None admin] RBAC: Adding query filter params (interface=public, service_id=6cd3e09be3dd44feb23e7211ab24ebec, region_id=RegionOne) {{(pid=88065) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:37:11.319841 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450c3e1d-c8a1-4136-a259-e6cd91f429bc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:37:11.321215 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450c3e1d-c8a1-4136-a259-e6cd91f429bc None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:37:11.326838 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 646/1300] 10.222.0.20 () {56 vars in 1251 bytes} [Thu Sep 22 02:37:11 2022] GET /identity/v3/endpoints?service_id=6cd3e09be3dd44feb23e7211ab24ebec&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:37:11.332617 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e30fefe-073a-4641-a476-51f0902fb569 None admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:37:11.420094 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:37:11.439052 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e30fefe-073a-4641-a476-51f0902fb569 None admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:37:11.473343 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d42efa8-d4c1-4c04-871d-7a9eaf40a604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:37:11.473958 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42efa8-d4c1-4c04-871d-7a9eaf40a604 None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:37:11.474237 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42efa8-d4c1-4c04-871d-7a9eaf40a604 None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:37:11.474511 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42efa8-d4c1-4c04-871d-7a9eaf40a604 None admin] PATH_INFO: `/v3/services` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:37:11.475145 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d42efa8-d4c1-4c04-871d-7a9eaf40a604 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:37:11.476507 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d42efa8-d4c1-4c04-871d-7a9eaf40a604 None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:37:11.483243 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 655/1301] 10.222.0.20 () {56 vars in 1087 bytes} [Thu Sep 22 02:37:11 2022] GET /identity/v3/services => generated 2191 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 02:37:13.655990 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685f13d-8a4a-4e47-a195-28c60eb1091f None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:37:13.656550 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685f13d-8a4a-4e47-a195-28c60eb1091f None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:37:13.656550 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685f13d-8a4a-4e47-a195-28c60eb1091f None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:37:13.657915 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 647/1302] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:37:13.663953 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7b49b-5553-445c-b55b-0847c38a171a None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:37:13.664485 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7b49b-5553-445c-b55b-0847c38a171a None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:37:13.664577 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7b49b-5553-445c-b55b-0847c38a171a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:37:13.683821 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-13b7b49b-5553-445c-b55b-0847c38a171a None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:37:13.685110 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-13b7b49b-5553-445c-b55b-0847c38a171a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:37:13.755766 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 656/1303] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:37:13 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 02:37:14.026299 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2685f13d-8a4a-4e47-a195-28c60eb1091f None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:37:14.126325 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:37:14.135108 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2685f13d-8a4a-4e47-a195-28c60eb1091f None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:37:14.172130 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aea9b2-0e1c-4fe6-ac6c-e6d94bf4a286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:37:14.172778 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aea9b2-0e1c-4fe6-ac6c-e6d94bf4a286 None admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:37:14.173065 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aea9b2-0e1c-4fe6-ac6c-e6d94bf4a286 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:37:14.173340 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aea9b2-0e1c-4fe6-ac6c-e6d94bf4a286 None admin] PATH_INFO: `/v3/services/placement` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:37:14.173931 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aea9b2-0e1c-4fe6-ac6c-e6d94bf4a286 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:37:14.175353 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aea9b2-0e1c-4fe6-ac6c-e6d94bf4a286 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:37:14.184489 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application [None req-21aea9b2-0e1c-4fe6-ac6c-e6d94bf4a286 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 02:37:14.184489 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:37:14.184489 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 02:37:14.184489 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 02:37:14.184489 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 02:37:14.184489 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 02:37:14.184489 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 02:37:14.184489 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:37:14.184489 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 02:37:14.184489 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:37:14.184489 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 02:37:14.184489 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:37:14.184489 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 02:37:14.184489 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 02:37:14.184489 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 02:37:14.184489 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 02:37:14.184489 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 02:37:14.185719 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 02:37:14.185719 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 02:37:14.185719 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 02:37:14.185719 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 02:37:14.185719 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 02:37:14.185719 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 02:37:14.185719 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 02:37:14.185719 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 02:37:14.185719 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 02:37:14.185719 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 02:37:14.185719 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 02:37:14.185719 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 02:37:14.185719 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 02:37:14.185719 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 02:37:14.185719 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 02:37:14.185719 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 02:37:14.186618 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 02:37:14.186618 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application with Lock( Sep 22 02:37:14.186618 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 02:37:14.186618 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self._enter() Sep 22 02:37:14.186618 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 02:37:14.186618 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 02:37:14.186618 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 02:37:14.186618 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application return self.creator() Sep 22 02:37:14.186618 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 02:37:14.186618 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application created_value = creator( Sep 22 02:37:14.186618 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 02:37:14.186618 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 02:37:14.186618 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 02:37:14.186618 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: ERROR keystone.server.flask.application Sep 22 02:37:14.186618 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 648/1304] 10.222.0.20 () {56 vars in 1117 bytes} [Thu Sep 22 02:37:14 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 02:37:14.191956 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7b49b-5553-445c-b55b-0847c38a171a None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:37:14.269906 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:37:14.294946 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7b49b-5553-445c-b55b-0847c38a171a None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:37:14.335946 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3693d0-6a2c-4c3b-bf4f-deb7b3adfd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:37:14.336901 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3693d0-6a2c-4c3b-bf4f-deb7b3adfd4c None admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:37:14.337694 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3693d0-6a2c-4c3b-bf4f-deb7b3adfd4c None admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:37:14.337970 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3693d0-6a2c-4c3b-bf4f-deb7b3adfd4c None admin] PATH_INFO: `/v3/services` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:37:14.338628 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3693d0-6a2c-4c3b-bf4f-deb7b3adfd4c None admin] RBAC: Adding query filter params (name=placement) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:37:14.338971 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3693d0-6a2c-4c3b-bf4f-deb7b3adfd4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:37:14.340406 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3693d0-6a2c-4c3b-bf4f-deb7b3adfd4c None admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:37:14.346501 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 657/1305] 10.222.0.20 () {56 vars in 1116 bytes} [Thu Sep 22 02:37:14 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:37:14.353039 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aea9b2-0e1c-4fe6-ac6c-e6d94bf4a286 None admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:37:14.466068 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aea9b2-0e1c-4fe6-ac6c-e6d94bf4a286 None admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:37:14.504929 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362eab6c-39f8-4ad2-8a4d-92ccb80410b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:37:14.505557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362eab6c-39f8-4ad2-8a4d-92ccb80410b5 None admin] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:37:14.505838 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362eab6c-39f8-4ad2-8a4d-92ccb80410b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:37:14.506111 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362eab6c-39f8-4ad2-8a4d-92ccb80410b5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:37:14.506796 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362eab6c-39f8-4ad2-8a4d-92ccb80410b5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:37:14.508221 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362eab6c-39f8-4ad2-8a4d-92ccb80410b5 None admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:37:14.530233 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 649/1306] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Sep 22 02:37:14 2022] POST /identity/v3/endpoints => generated 331 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:37:24.343716 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=30 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:37:24.525237 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:38:23.572301 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518e389-2d69-4f53-a40e-e09266e11a8b None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:38:23.572794 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518e389-2d69-4f53-a40e-e09266e11a8b None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:38:23.573267 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518e389-2d69-4f53-a40e-e09266e11a8b None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:38:23.581941 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3eb41-664d-4bac-8d8d-c8fb6d8b7763 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:38:23.582419 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3eb41-664d-4bac-8d8d-c8fb6d8b7763 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:38:23.582861 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3eb41-664d-4bac-8d8d-c8fb6d8b7763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:38:23.585305 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 658/1307] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:38:23 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:38:23.692487 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-03f3eb41-664d-4bac-8d8d-c8fb6d8b7763 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:38:23.694463 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-03f3eb41-664d-4bac-8d8d-c8fb6d8b7763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:38:23.836731 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 650/1308] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:38:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:38:24.133683 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de4b2d-97b3-47a4-a6e4-1749a7561997 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:38:24.134532 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de4b2d-97b3-47a4-a6e4-1749a7561997 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:38:24.135082 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de4b2d-97b3-47a4-a6e4-1749a7561997 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:38:24.136936 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 659/1309] 10.222.0.20 () {54 vars in 963 bytes} [Thu Sep 22 02:38:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:38:24.143681 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560857d9-d053-441c-af24-9ff3f221977f None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:38:24.144000 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560857d9-d053-441c-af24-9ff3f221977f None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:38:24.144231 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560857d9-d053-441c-af24-9ff3f221977f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:38:24.246907 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-560857d9-d053-441c-af24-9ff3f221977f None None] MFA Rules not processed for user `a0f635200ad241b3b31ed49701bc511b`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:38:24.248192 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-560857d9-d053-441c-af24-9ff3f221977f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:38:24.276957 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:38:24.351324 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 651/1310] 10.222.0.20 () {58 vars in 1060 bytes} [Thu Sep 22 02:38:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:38:24.361096 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de4b2d-97b3-47a4-a6e4-1749a7561997 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:38:24.362639 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-d0de4b2d-97b3-47a4-a6e4-1749a7561997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:38:24.513743 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de4b2d-97b3-47a4-a6e4-1749a7561997 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:38:24.624136 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f496a-3e34-4321-8fde-663b00e42f0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f635200ad241b3b31ed49701bc511b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:38:24.624771 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f496a-3e34-4321-8fde-663b00e42f0f service neutron] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:38:24.625099 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f496a-3e34-4321-8fde-663b00e42f0f service neutron] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:38:24.625378 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f496a-3e34-4321-8fde-663b00e42f0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:38:24.627402 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-d02f496a-3e34-4321-8fde-663b00e42f0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:38:24.701134 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:38:24.772212 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02f496a-3e34-4321-8fde-663b00e42f0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:38:24.773703 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02f496a-3e34-4321-8fde-663b00e42f0f service neutron] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:38:24.794715 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 660/1311] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:38:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:38:27.497993 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070c72c-9d71-4599-ba77-6a4dad683752 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:38:27.498538 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070c72c-9d71-4599-ba77-6a4dad683752 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:38:27.498628 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070c72c-9d71-4599-ba77-6a4dad683752 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:38:27.499985 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 652/1312] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:38:27.506193 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379741d-5af6-4d9d-a197-4c1beea9466e None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:38:27.506642 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379741d-5af6-4d9d-a197-4c1beea9466e None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:38:27.506769 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379741d-5af6-4d9d-a197-4c1beea9466e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:38:27.606959 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-8379741d-5af6-4d9d-a197-4c1beea9466e None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:38:27.608131 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-8379741d-5af6-4d9d-a197-4c1beea9466e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:38:27.713414 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 661/1313] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:38:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:38:28.027743 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f2820-d268-460b-95e5-af91c314d3b7 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:38:28.028032 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f2820-d268-460b-95e5-af91c314d3b7 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:38:28.028308 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f2820-d268-460b-95e5-af91c314d3b7 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:38:28.029658 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 653/1314] 10.222.0.20 () {54 vars in 963 bytes} [Thu Sep 22 02:38:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:38:28.035684 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b07def-aecd-4647-be57-0eb06444e071 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:38:28.036553 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b07def-aecd-4647-be57-0eb06444e071 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:38:28.036834 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b07def-aecd-4647-be57-0eb06444e071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:38:28.048561 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:38:28.117438 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-a0b07def-aecd-4647-be57-0eb06444e071 None None] MFA Rules not processed for user `a0f635200ad241b3b31ed49701bc511b`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:38:28.118676 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-a0b07def-aecd-4647-be57-0eb06444e071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:38:28.197588 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 662/1315] 10.222.0.20 () {58 vars in 1060 bytes} [Thu Sep 22 02:38:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:38:28.206811 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f2820-d268-460b-95e5-af91c314d3b7 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:38:28.208953 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-612f2820-d268-460b-95e5-af91c314d3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:38:28.324906 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f2820-d268-460b-95e5-af91c314d3b7 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:38:28.410111 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:38:28.414871 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085e3225-0bda-425d-bcda-6ff1bd1cb1ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f635200ad241b3b31ed49701bc511b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:38:28.415561 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e3225-0bda-425d-bcda-6ff1bd1cb1ee service neutron] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:38:28.415846 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e3225-0bda-425d-bcda-6ff1bd1cb1ee service neutron] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:38:28.416123 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e3225-0bda-425d-bcda-6ff1bd1cb1ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:38:28.417852 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-085e3225-0bda-425d-bcda-6ff1bd1cb1ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:38:28.538341 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085e3225-0bda-425d-bcda-6ff1bd1cb1ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:38:28.539841 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085e3225-0bda-425d-bcda-6ff1bd1cb1ee service neutron] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:38:28.558142 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 654/1316] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:38:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:38:31.263630 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c303d8-ae7e-4b24-a8d2-b9d1a7bd0172 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:38:31.264198 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c303d8-ae7e-4b24-a8d2-b9d1a7bd0172 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:38:31.264198 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c303d8-ae7e-4b24-a8d2-b9d1a7bd0172 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:38:31.265627 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 663/1317] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:38:31.272051 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2edcc-3007-4404-ac3c-bcb2242b4d45 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:38:31.272558 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2edcc-3007-4404-ac3c-bcb2242b4d45 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:38:31.272791 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2edcc-3007-4404-ac3c-bcb2242b4d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:38:31.343819 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-aae2edcc-3007-4404-ac3c-bcb2242b4d45 None None] MFA Rules not processed for user `c651e80ba10343f4ab505b7e2c378648`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:38:31.345089 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-aae2edcc-3007-4404-ac3c-bcb2242b4d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:38:31.403579 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:38:31.448669 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 655/1318] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:38:31 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:38:31.742640 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c303d8-ae7e-4b24-a8d2-b9d1a7bd0172 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:38:31.744199 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-58c303d8-ae7e-4b24-a8d2-b9d1a7bd0172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:38:31.850750 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c303d8-ae7e-4b24-a8d2-b9d1a7bd0172 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:38:31.853739 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:38:31.931666 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f5aac6-3723-4ddb-9ebd-c2f16609032b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f635200ad241b3b31ed49701bc511b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:38:31.932297 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5aac6-3723-4ddb-9ebd-c2f16609032b service neutron] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:38:31.932579 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5aac6-3723-4ddb-9ebd-c2f16609032b service neutron] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:38:31.932851 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5aac6-3723-4ddb-9ebd-c2f16609032b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:38:31.934627 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-19f5aac6-3723-4ddb-9ebd-c2f16609032b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:38:32.063615 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f5aac6-3723-4ddb-9ebd-c2f16609032b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:38:32.065112 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f5aac6-3723-4ddb-9ebd-c2f16609032b service neutron] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:38:32.086395 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 664/1319] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:38:31 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:38:35.038410 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98370b14-137a-4ab2-8b86-47d8229ba5b9 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:38:35.038906 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98370b14-137a-4ab2-8b86-47d8229ba5b9 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:38:35.039040 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98370b14-137a-4ab2-8b86-47d8229ba5b9 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:38:35.040366 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 656/1320] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:38:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:38:35.045892 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319e42f-a1e0-4878-9dd5-2fd50726e9c1 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:38:35.046421 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319e42f-a1e0-4878-9dd5-2fd50726e9c1 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:38:35.046421 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319e42f-a1e0-4878-9dd5-2fd50726e9c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:38:35.091832 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:38:35.121659 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-6319e42f-a1e0-4878-9dd5-2fd50726e9c1 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:38:35.122972 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-6319e42f-a1e0-4878-9dd5-2fd50726e9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:38:35.227531 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 665/1321] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:38:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:38:35.521024 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98370b14-137a-4ab2-8b86-47d8229ba5b9 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:38:35.522858 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-98370b14-137a-4ab2-8b86-47d8229ba5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:38:35.628795 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98370b14-137a-4ab2-8b86-47d8229ba5b9 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:38:35.706892 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88e12a-3912-4226-a9ab-1edc757fb4f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f635200ad241b3b31ed49701bc511b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:38:35.707580 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88e12a-3912-4226-a9ab-1edc757fb4f6 service neutron] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:38:35.707863 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88e12a-3912-4226-a9ab-1edc757fb4f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:38:35.708134 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88e12a-3912-4226-a9ab-1edc757fb4f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:38:35.709740 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-8e88e12a-3912-4226-a9ab-1edc757fb4f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:38:35.716843 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:38:35.820722 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e88e12a-3912-4226-a9ab-1edc757fb4f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:38:35.822160 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e88e12a-3912-4226-a9ab-1edc757fb4f6 service neutron] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:38:35.840155 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 657/1322] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:38:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:38:38.801642 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7f6ca-e583-4696-aa4a-b3d3fbf008c8 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:38:38.802290 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7f6ca-e583-4696-aa4a-b3d3fbf008c8 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:38:38.802290 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7f6ca-e583-4696-aa4a-b3d3fbf008c8 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:38:38.803564 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 666/1323] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:38:38.809196 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46ad33-4550-47b2-99bb-bc9dc5861665 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:38:38.809683 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46ad33-4550-47b2-99bb-bc9dc5861665 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:38:38.809804 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46ad33-4550-47b2-99bb-bc9dc5861665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:38:38.887651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-1d46ad33-4550-47b2-99bb-bc9dc5861665 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:38:38.888933 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-1d46ad33-4550-47b2-99bb-bc9dc5861665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:38:38.959735 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:38:39.008510 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 658/1324] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:38:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 02:38:39.293329 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa7f6ca-e583-4696-aa4a-b3d3fbf008c8 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:38:39.294926 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-4aa7f6ca-e583-4696-aa4a-b3d3fbf008c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:38:39.442372 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa7f6ca-e583-4696-aa4a-b3d3fbf008c8 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:38:39.473512 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:38:39.545830 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af478e-4841-44f0-a13c-eb19d10eb83f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f635200ad241b3b31ed49701bc511b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:38:39.546537 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af478e-4841-44f0-a13c-eb19d10eb83f service neutron] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:38:39.547051 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af478e-4841-44f0-a13c-eb19d10eb83f service neutron] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:38:39.547446 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af478e-4841-44f0-a13c-eb19d10eb83f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:38:39.549215 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-36af478e-4841-44f0-a13c-eb19d10eb83f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:38:39.679770 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af478e-4841-44f0-a13c-eb19d10eb83f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:38:39.681239 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af478e-4841-44f0-a13c-eb19d10eb83f service neutron] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:38:39.703954 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 667/1325] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:38:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:38:42.001285 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda65fd-0aac-4342-a656-9f3006c89576 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:38:42.001848 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda65fd-0aac-4342-a656-9f3006c89576 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:38:42.001848 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda65fd-0aac-4342-a656-9f3006c89576 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:38:42.003230 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 659/1326] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:38:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:38:42.008798 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc22cd3-0768-451c-b658-6caaabbd551c None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:38:42.009251 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc22cd3-0768-451c-b658-6caaabbd551c None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:38:42.009488 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc22cd3-0768-451c-b658-6caaabbd551c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:38:42.068467 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:38:42.090254 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-afc22cd3-0768-451c-b658-6caaabbd551c None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:38:42.091528 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-afc22cd3-0768-451c-b658-6caaabbd551c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:38:42.201886 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 668/1327] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:38:42.484803 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda65fd-0aac-4342-a656-9f3006c89576 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:38:42.486382 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-ccda65fd-0aac-4342-a656-9f3006c89576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:38:42.608333 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda65fd-0aac-4342-a656-9f3006c89576 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:38:42.686614 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b31a0f-084b-4bf6-bd5e-4b3b9a7c5a70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f635200ad241b3b31ed49701bc511b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:38:42.687258 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b31a0f-084b-4bf6-bd5e-4b3b9a7c5a70 service neutron] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:38:42.687538 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b31a0f-084b-4bf6-bd5e-4b3b9a7c5a70 service neutron] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:38:42.687809 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b31a0f-084b-4bf6-bd5e-4b3b9a7c5a70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:38:42.689435 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-78b31a0f-084b-4bf6-bd5e-4b3b9a7c5a70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:38:42.707984 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:38:42.802641 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b31a0f-084b-4bf6-bd5e-4b3b9a7c5a70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:38:42.804029 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b31a0f-084b-4bf6-bd5e-4b3b9a7c5a70 service neutron] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:38:42.825229 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 660/1328] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:38:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 02:38:45.058411 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebfb89f-ef0b-4d45-b459-3e0cfaaf2959 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:38:45.058998 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebfb89f-ef0b-4d45-b459-3e0cfaaf2959 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:38:45.059097 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebfb89f-ef0b-4d45-b459-3e0cfaaf2959 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:38:45.060429 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 669/1329] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:38:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:38:45.065604 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46052424-8a11-4c51-ad56-65765ca975e0 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:38:45.066020 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46052424-8a11-4c51-ad56-65765ca975e0 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:38:45.066163 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46052424-8a11-4c51-ad56-65765ca975e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:38:45.133659 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-46052424-8a11-4c51-ad56-65765ca975e0 None None] MFA Rules not processed for user `c651e80ba10343f4ab505b7e2c378648`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:38:45.134817 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-46052424-8a11-4c51-ad56-65765ca975e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:38:45.200392 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:38:45.232846 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 661/1330] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:38:45 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:38:45.521271 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebfb89f-ef0b-4d45-b459-3e0cfaaf2959 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:38:45.522864 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-5ebfb89f-ef0b-4d45-b459-3e0cfaaf2959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:38:45.642478 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebfb89f-ef0b-4d45-b459-3e0cfaaf2959 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:38:45.666500 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:38:45.740273 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156ba58-20f5-4b33-9326-b603b6de96bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f635200ad241b3b31ed49701bc511b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:38:45.740898 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156ba58-20f5-4b33-9326-b603b6de96bc service neutron] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:38:45.741185 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156ba58-20f5-4b33-9326-b603b6de96bc service neutron] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:38:45.741466 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156ba58-20f5-4b33-9326-b603b6de96bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:38:45.743123 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-8156ba58-20f5-4b33-9326-b603b6de96bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:38:45.859083 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156ba58-20f5-4b33-9326-b603b6de96bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:38:45.860563 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156ba58-20f5-4b33-9326-b603b6de96bc service neutron] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:38:45.879817 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 670/1331] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:38:45 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:38:48.305408 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e3d65-5070-43e5-aadf-5fe5ec006d1e None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:38:48.305947 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e3d65-5070-43e5-aadf-5fe5ec006d1e None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:38:48.305947 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e3d65-5070-43e5-aadf-5fe5ec006d1e None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:38:48.307517 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 662/1332] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:38:48.314649 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e3dea-66c5-4ce1-ae71-f698a01eb5c8 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:38:48.315258 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e3dea-66c5-4ce1-ae71-f698a01eb5c8 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:38:48.315258 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e3dea-66c5-4ce1-ae71-f698a01eb5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:38:48.390409 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:38:48.394430 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-704e3dea-66c5-4ce1-ae71-f698a01eb5c8 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:38:48.396539 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-704e3dea-66c5-4ce1-ae71-f698a01eb5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:38:48.502209 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 671/1333] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:38:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:38:48.787699 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e3d65-5070-43e5-aadf-5fe5ec006d1e None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:38:48.789266 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-0e7e3d65-5070-43e5-aadf-5fe5ec006d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:38:48.894591 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e3d65-5070-43e5-aadf-5fe5ec006d1e None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:38:48.975856 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee4fe9-edc9-4947-91af-ea25eccf15ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f635200ad241b3b31ed49701bc511b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:38:48.976465 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee4fe9-edc9-4947-91af-ea25eccf15ff service neutron] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:38:48.976751 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee4fe9-edc9-4947-91af-ea25eccf15ff service neutron] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:38:48.977030 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee4fe9-edc9-4947-91af-ea25eccf15ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:38:48.978660 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-0aee4fe9-edc9-4947-91af-ea25eccf15ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:38:49.006911 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:38:49.090241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aee4fe9-edc9-4947-91af-ea25eccf15ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:38:49.091737 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aee4fe9-edc9-4947-91af-ea25eccf15ff service neutron] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:38:49.109280 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 663/1334] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:38:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:38:51.333794 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fca430-0ad2-45e5-8f43-6a9e9d1c5257 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:38:51.334326 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fca430-0ad2-45e5-8f43-6a9e9d1c5257 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:38:51.334441 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fca430-0ad2-45e5-8f43-6a9e9d1c5257 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:38:51.335736 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 672/1335] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:38:51.340902 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed18818-6134-4c1a-b118-add2f241c2f1 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:38:51.341380 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed18818-6134-4c1a-b118-add2f241c2f1 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:38:51.341476 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed18818-6134-4c1a-b118-add2f241c2f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:38:51.409193 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-6ed18818-6134-4c1a-b118-add2f241c2f1 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:38:51.411041 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-6ed18818-6134-4c1a-b118-add2f241c2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:38:51.494084 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:38:51.516672 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 664/1336] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:38:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:38:51.805006 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fca430-0ad2-45e5-8f43-6a9e9d1c5257 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:38:51.806566 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-c0fca430-0ad2-45e5-8f43-6a9e9d1c5257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:38:51.913524 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fca430-0ad2-45e5-8f43-6a9e9d1c5257 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:38:51.944569 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:38:51.992334 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baf3ad8-1554-405e-80f5-37ef70db2692 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f635200ad241b3b31ed49701bc511b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:38:51.992944 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf3ad8-1554-405e-80f5-37ef70db2692 service neutron] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:38:51.993241 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf3ad8-1554-405e-80f5-37ef70db2692 service neutron] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:38:51.993503 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf3ad8-1554-405e-80f5-37ef70db2692 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:38:51.995153 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-1baf3ad8-1554-405e-80f5-37ef70db2692 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:38:52.114073 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baf3ad8-1554-405e-80f5-37ef70db2692 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:38:52.115555 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baf3ad8-1554-405e-80f5-37ef70db2692 service neutron] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:38:52.135134 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 673/1337] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:38:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:38:54.735607 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb58f0-aa03-408e-a606-942194929cb9 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:38:54.736383 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb58f0-aa03-408e-a606-942194929cb9 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:38:54.736383 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb58f0-aa03-408e-a606-942194929cb9 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:38:54.737668 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 665/1338] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:38:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:38:54.743350 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798dc1de-5023-41aa-9091-ee0cbb3584ec None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:38:54.743882 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798dc1de-5023-41aa-9091-ee0cbb3584ec None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:38:54.743882 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798dc1de-5023-41aa-9091-ee0cbb3584ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:38:54.836603 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-798dc1de-5023-41aa-9091-ee0cbb3584ec None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:38:54.838347 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-798dc1de-5023-41aa-9091-ee0cbb3584ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:38:54.849276 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:38:54.947481 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 674/1339] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:38:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:38:55.234706 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb58f0-aa03-408e-a606-942194929cb9 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:38:55.236288 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-41cb58f0-aa03-408e-a606-942194929cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:38:55.352908 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb58f0-aa03-408e-a606-942194929cb9 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:38:55.437063 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993fbb4-a37e-471e-9854-e311fd1f7443 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f635200ad241b3b31ed49701bc511b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:38:55.437703 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993fbb4-a37e-471e-9854-e311fd1f7443 service neutron] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:38:55.437982 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993fbb4-a37e-471e-9854-e311fd1f7443 service neutron] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:38:55.438250 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993fbb4-a37e-471e-9854-e311fd1f7443 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:38:55.439933 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-7993fbb4-a37e-471e-9854-e311fd1f7443 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:38:55.478446 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:38:55.571133 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7993fbb4-a37e-471e-9854-e311fd1f7443 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:38:55.572592 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7993fbb4-a37e-471e-9854-e311fd1f7443 service neutron] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:38:55.593631 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 666/1340] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:38:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:39:00.038553 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d534d2-213c-48aa-a5af-bb6672fd9fde None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:00.039480 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d534d2-213c-48aa-a5af-bb6672fd9fde None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:00.039480 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d534d2-213c-48aa-a5af-bb6672fd9fde None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:00.040547 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 675/1341] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:39:00.046751 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e249ab24-085a-4a88-b849-34dc5e3c746b None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:00.047374 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e249ab24-085a-4a88-b849-34dc5e3c746b None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:00.047374 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e249ab24-085a-4a88-b849-34dc5e3c746b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:00.127555 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-e249ab24-085a-4a88-b849-34dc5e3c746b None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:39:00.128667 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-e249ab24-085a-4a88-b849-34dc5e3c746b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:00.232552 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:00.248029 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 667/1342] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:39:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:39:00.535664 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d534d2-213c-48aa-a5af-bb6672fd9fde None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:39:00.541730 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-d8d534d2-213c-48aa-a5af-bb6672fd9fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:00.672587 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d534d2-213c-48aa-a5af-bb6672fd9fde None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:39:00.741645 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:00.791252 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60834075-7438-440e-aee9-f569a50457ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f635200ad241b3b31ed49701bc511b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:39:00.791755 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60834075-7438-440e-aee9-f569a50457ba service neutron] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:00.792035 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60834075-7438-440e-aee9-f569a50457ba service neutron] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:00.792306 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60834075-7438-440e-aee9-f569a50457ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:00.793941 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-60834075-7438-440e-aee9-f569a50457ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:00.914697 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60834075-7438-440e-aee9-f569a50457ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:39:00.916412 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60834075-7438-440e-aee9-f569a50457ba service neutron] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:39:00.938232 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 676/1343] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:39:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:39:04.115250 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572ea6d-e909-4cf4-9914-8667f033143b None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:04.115819 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572ea6d-e909-4cf4-9914-8667f033143b None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:04.115819 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572ea6d-e909-4cf4-9914-8667f033143b None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:04.117157 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 668/1344] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:39:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:39:04.123293 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82006ace-ba45-40d9-9575-0d5367f8e58d None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:04.125407 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82006ace-ba45-40d9-9575-0d5367f8e58d None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:04.125407 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82006ace-ba45-40d9-9575-0d5367f8e58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:04.203340 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-82006ace-ba45-40d9-9575-0d5367f8e58d None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:39:04.204579 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-82006ace-ba45-40d9-9575-0d5367f8e58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:04.225305 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:04.319205 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 677/1345] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:39:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 02:39:04.612323 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b572ea6d-e909-4cf4-9914-8667f033143b None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:39:04.614125 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-b572ea6d-e909-4cf4-9914-8667f033143b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:04.740576 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b572ea6d-e909-4cf4-9914-8667f033143b None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:39:04.849671 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae378050-4318-4783-a29b-9aabce264f36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f635200ad241b3b31ed49701bc511b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:39:04.851494 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae378050-4318-4783-a29b-9aabce264f36 service neutron] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:04.851967 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae378050-4318-4783-a29b-9aabce264f36 service neutron] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:04.852881 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae378050-4318-4783-a29b-9aabce264f36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:04.854440 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-ae378050-4318-4783-a29b-9aabce264f36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:04.908959 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:05.014554 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae378050-4318-4783-a29b-9aabce264f36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:39:05.016089 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae378050-4318-4783-a29b-9aabce264f36 service neutron] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:39:05.038941 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 669/1346] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:39:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:39:09.668843 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915f89f-7b6b-4c39-bb7f-eeb9e4f1a7fc None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:09.669547 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915f89f-7b6b-4c39-bb7f-eeb9e4f1a7fc None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:09.669547 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915f89f-7b6b-4c39-bb7f-eeb9e4f1a7fc None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:09.674945 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 678/1347] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:39:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:39:09.678999 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5d3c6-8502-43b9-b6fa-45cb02d5fdb4 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:09.679507 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5d3c6-8502-43b9-b6fa-45cb02d5fdb4 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:09.679995 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5d3c6-8502-43b9-b6fa-45cb02d5fdb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:09.808105 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-8ab5d3c6-8502-43b9-b6fa-45cb02d5fdb4 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:39:09.809485 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-8ab5d3c6-8502-43b9-b6fa-45cb02d5fdb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:09.944909 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:09.947681 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 670/1348] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:39:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:39:10.242265 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915f89f-7b6b-4c39-bb7f-eeb9e4f1a7fc None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:39:10.244076 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-c915f89f-7b6b-4c39-bb7f-eeb9e4f1a7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:10.368935 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915f89f-7b6b-4c39-bb7f-eeb9e4f1a7fc None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:39:10.423731 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:10.454076 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1f132-2199-4cda-9950-2d87977a6638 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f635200ad241b3b31ed49701bc511b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:39:10.454698 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1f132-2199-4cda-9950-2d87977a6638 service neutron] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:10.455017 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1f132-2199-4cda-9950-2d87977a6638 service neutron] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:10.455292 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1f132-2199-4cda-9950-2d87977a6638 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:10.457603 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-4bb1f132-2199-4cda-9950-2d87977a6638 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:10.579126 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb1f132-2199-4cda-9950-2d87977a6638 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:39:10.580589 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb1f132-2199-4cda-9950-2d87977a6638 service neutron] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:39:10.601843 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 679/1349] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:39:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 02:39:15.973356 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec79efb-dad9-4e64-aa9a-4e1e72197b9f None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:15.974346 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec79efb-dad9-4e64-aa9a-4e1e72197b9f None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:15.974769 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec79efb-dad9-4e64-aa9a-4e1e72197b9f None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:15.976769 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 671/1350] 10.222.0.20 () {54 vars in 921 bytes} [Thu Sep 22 02:39:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:39:15.982739 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30b78d-560d-40d7-9fd3-740101bd023c None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:15.983685 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30b78d-560d-40d7-9fd3-740101bd023c None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:15.984144 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30b78d-560d-40d7-9fd3-740101bd023c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:16.138577 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-be30b78d-560d-40d7-9fd3-740101bd023c None None] MFA Rules not processed for user `865576e9e17d4efdbd1ca6a005f31bda`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:39:16.141629 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-be30b78d-560d-40d7-9fd3-740101bd023c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:16.191083 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:16.352868 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 680/1351] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Sep 22 02:39:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 02:39:21.597597 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c1ed57-9be6-412f-8bcd-4992a959d9a5 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:21.599287 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c1ed57-9be6-412f-8bcd-4992a959d9a5 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:21.600046 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c1ed57-9be6-412f-8bcd-4992a959d9a5 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:21.602673 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 672/1352] 10.222.0.20 () {54 vars in 921 bytes} [Thu Sep 22 02:39:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:39:21.608552 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5bdac-97c5-4c44-ae9e-fa4c98fcd533 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:21.609129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5bdac-97c5-4c44-ae9e-fa4c98fcd533 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:21.609129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5bdac-97c5-4c44-ae9e-fa4c98fcd533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:21.693839 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-8cc5bdac-97c5-4c44-ae9e-fa4c98fcd533 None None] MFA Rules not processed for user `865576e9e17d4efdbd1ca6a005f31bda`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:39:21.695002 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-8cc5bdac-97c5-4c44-ae9e-fa4c98fcd533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:21.775679 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:21.817423 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 681/1353] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Sep 22 02:39:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 02:39:23.944575 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512077ab-ea4b-4cad-a02b-bbd2d4670120 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:23.945399 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512077ab-ea4b-4cad-a02b-bbd2d4670120 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:23.945399 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512077ab-ea4b-4cad-a02b-bbd2d4670120 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:23.946465 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 673/1354] 10.222.0.20 () {54 vars in 921 bytes} [Thu Sep 22 02:39:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:39:23.953234 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b97e10-9a75-41b3-aef3-edf8cc48d6df None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:23.953971 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b97e10-9a75-41b3-aef3-edf8cc48d6df None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:23.953971 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b97e10-9a75-41b3-aef3-edf8cc48d6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:24.046890 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-95b97e10-9a75-41b3-aef3-edf8cc48d6df None None] MFA Rules not processed for user `865576e9e17d4efdbd1ca6a005f31bda`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:39:24.048103 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-95b97e10-9a75-41b3-aef3-edf8cc48d6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:24.172567 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 682/1355] 10.222.0.20 () {58 vars in 1018 bytes} [Thu Sep 22 02:39:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:39:27.901763 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2879f9-7497-48e1-af34-be1b4859eabf None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:27.903706 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2879f9-7497-48e1-af34-be1b4859eabf None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:27.903706 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2879f9-7497-48e1-af34-be1b4859eabf None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:27.903706 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 674/1356] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:39:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:39:27.909500 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b612a-d40c-46b8-bea3-7b1202ff692d None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:27.909978 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b612a-d40c-46b8-bea3-7b1202ff692d None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:27.910098 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b612a-d40c-46b8-bea3-7b1202ff692d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:27.962674 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:28.014344 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-b42b612a-d40c-46b8-bea3-7b1202ff692d None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:39:28.016057 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-b42b612a-d40c-46b8-bea3-7b1202ff692d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:28.146559 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 683/1357] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:39:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:39:28.478707 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a859b72-9473-4d6e-8b6a-02777a287ecf None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:28.480157 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a859b72-9473-4d6e-8b6a-02777a287ecf None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:28.480954 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a859b72-9473-4d6e-8b6a-02777a287ecf None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:28.484557 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 675/1358] 10.222.0.20 () {54 vars in 965 bytes} [Thu Sep 22 02:39:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:39:28.497140 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250786e-e427-46c2-befb-44bb4c9a1c21 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:28.497937 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250786e-e427-46c2-befb-44bb4c9a1c21 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:28.498583 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250786e-e427-46c2-befb-44bb4c9a1c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:28.603622 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-f250786e-e427-46c2-befb-44bb4c9a1c21 None None] MFA Rules not processed for user `a64ed6c8e15d479887ab1dc59997d715`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:39:28.605261 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-f250786e-e427-46c2-befb-44bb4c9a1c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:28.681477 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:28.775864 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 684/1359] 10.222.0.20 () {58 vars in 1062 bytes} [Thu Sep 22 02:39:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:39:28.784205 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a859b72-9473-4d6e-8b6a-02777a287ecf None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:39:28.787780 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-9a859b72-9473-4d6e-8b6a-02777a287ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:28.969007 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a859b72-9473-4d6e-8b6a-02777a287ecf None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:39:29.078167 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:29.135837 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4946bba9-b51d-44fd-8701-ea4236b14cbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a64ed6c8e15d479887ab1dc59997d715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:39:29.136487 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4946bba9-b51d-44fd-8701-ea4236b14cbf service nova] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:29.136764 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4946bba9-b51d-44fd-8701-ea4236b14cbf service nova] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:29.137115 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4946bba9-b51d-44fd-8701-ea4236b14cbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:29.139388 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-4946bba9-b51d-44fd-8701-ea4236b14cbf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:29.276990 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4946bba9-b51d-44fd-8701-ea4236b14cbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:39:29.279186 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4946bba9-b51d-44fd-8701-ea4236b14cbf service nova] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:39:29.303997 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 676/1360] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:39:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 02:39:32.185889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c904c903-ecb9-470d-b433-410d0ee1f65f None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:32.186743 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c904c903-ecb9-470d-b433-410d0ee1f65f None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:32.186743 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c904c903-ecb9-470d-b433-410d0ee1f65f None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:32.187859 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 685/1361] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:39:32.197794 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c300dd-16b6-4a85-8f9f-11fd0ecc85ee None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:32.198290 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c300dd-16b6-4a85-8f9f-11fd0ecc85ee None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:32.198782 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c300dd-16b6-4a85-8f9f-11fd0ecc85ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:32.324251 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-68c300dd-16b6-4a85-8f9f-11fd0ecc85ee None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:39:32.326339 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-68c300dd-16b6-4a85-8f9f-11fd0ecc85ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:32.402979 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:32.528908 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 677/1362] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:39:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:39:32.821726 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c904c903-ecb9-470d-b433-410d0ee1f65f None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:39:32.823281 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-c904c903-ecb9-470d-b433-410d0ee1f65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:33.030770 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c904c903-ecb9-470d-b433-410d0ee1f65f None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:39:33.041926 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:33.161959 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354876cb-f6ba-476b-bb51-faa7b7b0ecbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a64ed6c8e15d479887ab1dc59997d715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:39:33.162580 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354876cb-f6ba-476b-bb51-faa7b7b0ecbc service nova] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:33.162889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354876cb-f6ba-476b-bb51-faa7b7b0ecbc service nova] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:33.163164 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354876cb-f6ba-476b-bb51-faa7b7b0ecbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:33.164811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-354876cb-f6ba-476b-bb51-faa7b7b0ecbc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:33.263905 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354876cb-f6ba-476b-bb51-faa7b7b0ecbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:39:33.265580 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354876cb-f6ba-476b-bb51-faa7b7b0ecbc service nova] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:39:33.286430 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 686/1363] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:39:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:39:35.871097 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6b66f-8de9-438e-93b0-edb0eb8dadb3 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:35.871718 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6b66f-8de9-438e-93b0-edb0eb8dadb3 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:35.872012 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6b66f-8de9-438e-93b0-edb0eb8dadb3 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:35.873669 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 678/1364] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:39:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:39:35.880128 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76407c33-c006-48ab-8b92-ad6f8457a676 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:35.880668 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76407c33-c006-48ab-8b92-ad6f8457a676 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:35.880668 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76407c33-c006-48ab-8b92-ad6f8457a676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:35.897878 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:35.960532 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-76407c33-c006-48ab-8b92-ad6f8457a676 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:39:35.961753 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-76407c33-c006-48ab-8b92-ad6f8457a676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:36.081303 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 687/1365] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:39:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:39:36.373812 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce57f8d-2c36-482b-b864-cda370eb2386 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:36.374257 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce57f8d-2c36-482b-b864-cda370eb2386 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:36.374634 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce57f8d-2c36-482b-b864-cda370eb2386 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:36.376251 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 679/1366] 10.222.0.20 () {54 vars in 965 bytes} [Thu Sep 22 02:39:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:39:36.383286 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b833c3-4f94-43b7-b77a-c619441fe64c None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:36.383670 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b833c3-4f94-43b7-b77a-c619441fe64c None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:36.383962 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b833c3-4f94-43b7-b77a-c619441fe64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:36.522788 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-97b833c3-4f94-43b7-b77a-c619441fe64c None None] MFA Rules not processed for user `a64ed6c8e15d479887ab1dc59997d715`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:39:36.525244 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-97b833c3-4f94-43b7-b77a-c619441fe64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:36.585284 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:36.763579 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 688/1367] 10.222.0.20 () {58 vars in 1062 bytes} [Thu Sep 22 02:39:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:39:36.773584 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce57f8d-2c36-482b-b864-cda370eb2386 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:39:36.777164 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-2ce57f8d-2c36-482b-b864-cda370eb2386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:36.981096 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce57f8d-2c36-482b-b864-cda370eb2386 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:39:37.006963 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:37.133068 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97fac5f-545b-406d-8c3c-d2ae3ac6909f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a64ed6c8e15d479887ab1dc59997d715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:39:37.133691 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97fac5f-545b-406d-8c3c-d2ae3ac6909f service nova] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:37.133978 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97fac5f-545b-406d-8c3c-d2ae3ac6909f service nova] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:37.134260 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97fac5f-545b-406d-8c3c-d2ae3ac6909f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:37.136047 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-f97fac5f-545b-406d-8c3c-d2ae3ac6909f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:37.143754 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88a1bd-4294-48e3-bd4c-8676b3533045 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:37.144693 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88a1bd-4294-48e3-bd4c-8676b3533045 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:37.144693 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88a1bd-4294-48e3-bd4c-8676b3533045 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:37.146368 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 689/1368] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:39:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:39:37.152694 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28a309-b4d1-4840-857d-129d369e2a9e None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:37.153490 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28a309-b4d1-4840-857d-129d369e2a9e None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:37.153859 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28a309-b4d1-4840-857d-129d369e2a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:37.240449 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-7d28a309-b4d1-4840-857d-129d369e2a9e None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:39:37.241860 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-7d28a309-b4d1-4840-857d-129d369e2a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:37.283756 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97fac5f-545b-406d-8c3c-d2ae3ac6909f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:39:37.285901 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97fac5f-545b-406d-8c3c-d2ae3ac6909f service nova] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:39:37.312416 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 680/1369] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:39:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 02:39:37.386193 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:37.396594 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 690/1370] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:39:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 02:39:37.713587 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71beab-119d-4cc3-8f49-dfbab763cb59 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:37.713851 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71beab-119d-4cc3-8f49-dfbab763cb59 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:37.714166 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71beab-119d-4cc3-8f49-dfbab763cb59 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:37.715782 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 681/1371] 10.222.0.20 () {54 vars in 963 bytes} [Thu Sep 22 02:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:39:37.721099 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e70e8-3727-4687-8818-45cb7ad19088 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:37.721496 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e70e8-3727-4687-8818-45cb7ad19088 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:37.721670 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e70e8-3727-4687-8818-45cb7ad19088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:37.825592 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-e25e70e8-3727-4687-8818-45cb7ad19088 None None] MFA Rules not processed for user `a08a0ed98bc14328a3d24309d7996b51`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:39:37.827072 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-e25e70e8-3727-4687-8818-45cb7ad19088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:37.972703 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 691/1372] 10.222.0.20 () {58 vars in 1060 bytes} [Thu Sep 22 02:39:37 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 02:39:37.980874 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71beab-119d-4cc3-8f49-dfbab763cb59 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:39:37.983813 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-af71beab-119d-4cc3-8f49-dfbab763cb59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:38.041385 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:38.183221 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71beab-119d-4cc3-8f49-dfbab763cb59 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:39:38.344895 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5585160-c0ba-4a25-b575-e764a8fea97c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08a0ed98bc14328a3d24309d7996b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:39:38.346514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5585160-c0ba-4a25-b575-e764a8fea97c service cinder] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:38.346514 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5585160-c0ba-4a25-b575-e764a8fea97c service cinder] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:38.348571 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5585160-c0ba-4a25-b575-e764a8fea97c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:38.352225 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-b5585160-c0ba-4a25-b575-e764a8fea97c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:38.512763 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:38.561601 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5585160-c0ba-4a25-b575-e764a8fea97c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:39:38.563101 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5585160-c0ba-4a25-b575-e764a8fea97c service cinder] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:39:38.583140 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 682/1373] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:39:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:39:39.681938 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917b7bd-6e89-47fe-a525-ce5db6a22bcf None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:39.682533 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917b7bd-6e89-47fe-a525-ce5db6a22bcf None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:39.682533 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917b7bd-6e89-47fe-a525-ce5db6a22bcf None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:39.683890 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 692/1374] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:39:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:39:39.689497 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9d55d-311a-4450-97f5-33d246261a17 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:39.690074 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9d55d-311a-4450-97f5-33d246261a17 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:39.690074 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9d55d-311a-4450-97f5-33d246261a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:39.775090 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-2da9d55d-311a-4450-97f5-33d246261a17 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:39:39.776332 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-2da9d55d-311a-4450-97f5-33d246261a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:39.920636 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 683/1375] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:39:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:39:40.234904 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917b7bd-6e89-47fe-a525-ce5db6a22bcf None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:39:40.236596 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-7917b7bd-6e89-47fe-a525-ce5db6a22bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:40.345355 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:40.435790 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917b7bd-6e89-47fe-a525-ce5db6a22bcf None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:39:40.597859 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2403e7c-40db-4a84-aed1-929583fd84e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a64ed6c8e15d479887ab1dc59997d715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:39:40.598485 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2403e7c-40db-4a84-aed1-929583fd84e2 service nova] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:40.598764 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2403e7c-40db-4a84-aed1-929583fd84e2 service nova] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:40.599073 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2403e7c-40db-4a84-aed1-929583fd84e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:40.600762 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-f2403e7c-40db-4a84-aed1-929583fd84e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:40.664953 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:40.720280 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2403e7c-40db-4a84-aed1-929583fd84e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:39:40.721923 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2403e7c-40db-4a84-aed1-929583fd84e2 service nova] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:39:40.756619 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 693/1376] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:39:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 02:39:43.286974 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e358e42-1b5e-4081-9655-904dd69264f6 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:43.287540 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e358e42-1b5e-4081-9655-904dd69264f6 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:43.287540 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e358e42-1b5e-4081-9655-904dd69264f6 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:43.288913 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 684/1377] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:39:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:39:43.294313 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ffaf28-0228-4c85-85c6-cec38f6115ac None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:43.294908 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ffaf28-0228-4c85-85c6-cec38f6115ac None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:43.294908 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ffaf28-0228-4c85-85c6-cec38f6115ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:43.375311 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-29ffaf28-0228-4c85-85c6-cec38f6115ac None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:39:43.376525 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-29ffaf28-0228-4c85-85c6-cec38f6115ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:43.486942 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 694/1378] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:39:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:39:43.780076 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e358e42-1b5e-4081-9655-904dd69264f6 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:39:43.782086 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-1e358e42-1b5e-4081-9655-904dd69264f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:43.829719 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:43.967441 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e358e42-1b5e-4081-9655-904dd69264f6 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:39:44.148677 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:44.156103 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d434e5bc-f108-457b-a3d9-ddbab7171469 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a64ed6c8e15d479887ab1dc59997d715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:39:44.157499 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434e5bc-f108-457b-a3d9-ddbab7171469 service nova] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:44.157814 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434e5bc-f108-457b-a3d9-ddbab7171469 service nova] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:44.158164 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434e5bc-f108-457b-a3d9-ddbab7171469 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:44.160176 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-d434e5bc-f108-457b-a3d9-ddbab7171469 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:44.309364 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d434e5bc-f108-457b-a3d9-ddbab7171469 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:39:44.312311 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d434e5bc-f108-457b-a3d9-ddbab7171469 service nova] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:39:44.350330 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 685/1379] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:39:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:39:46.706566 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506fdf2-88cb-4dce-bf86-fd4b13d74d3f None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:46.707333 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506fdf2-88cb-4dce-bf86-fd4b13d74d3f None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:46.707333 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506fdf2-88cb-4dce-bf86-fd4b13d74d3f None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:46.711562 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 695/1380] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:39:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:39:46.714529 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088497fd-1db5-45e8-b6c1-06202f0c1e39 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:46.715010 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088497fd-1db5-45e8-b6c1-06202f0c1e39 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:46.715244 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088497fd-1db5-45e8-b6c1-06202f0c1e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:46.795841 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-088497fd-1db5-45e8-b6c1-06202f0c1e39 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:39:46.797373 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-088497fd-1db5-45e8-b6c1-06202f0c1e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:46.884299 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:46.942541 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 686/1381] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:39:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:39:47.243887 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506fdf2-88cb-4dce-bf86-fd4b13d74d3f None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:39:47.245607 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-7506fdf2-88cb-4dce-bf86-fd4b13d74d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:47.287963 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:47.456455 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506fdf2-88cb-4dce-bf86-fd4b13d74d3f None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:39:47.576551 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:47.595611 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab725901-83b1-4548-8734-e6d92bbef0ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a64ed6c8e15d479887ab1dc59997d715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:39:47.596403 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab725901-83b1-4548-8734-e6d92bbef0ad service nova] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:47.596663 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab725901-83b1-4548-8734-e6d92bbef0ad service nova] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:47.596939 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab725901-83b1-4548-8734-e6d92bbef0ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:47.598657 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-ab725901-83b1-4548-8734-e6d92bbef0ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:47.699933 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab725901-83b1-4548-8734-e6d92bbef0ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:39:47.701611 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab725901-83b1-4548-8734-e6d92bbef0ad service nova] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:39:47.720635 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 696/1382] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:39:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:39:50.273342 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606da42-9b4e-47f9-9ef6-b1b81f348394 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:50.273919 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606da42-9b4e-47f9-9ef6-b1b81f348394 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:50.273919 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606da42-9b4e-47f9-9ef6-b1b81f348394 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:50.275531 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 687/1383] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:39:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:39:50.281404 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb5cfa-975b-4cd0-a59e-d4f28fb84795 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:50.281964 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb5cfa-975b-4cd0-a59e-d4f28fb84795 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:50.281964 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb5cfa-975b-4cd0-a59e-d4f28fb84795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:50.361826 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-aacb5cfa-975b-4cd0-a59e-d4f28fb84795 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:39:50.363136 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-aacb5cfa-975b-4cd0-a59e-d4f28fb84795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:50.433118 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:50.498238 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 697/1384] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:39:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:39:50.821471 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7606da42-9b4e-47f9-9ef6-b1b81f348394 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:39:50.823260 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-7606da42-9b4e-47f9-9ef6-b1b81f348394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:51.000595 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7606da42-9b4e-47f9-9ef6-b1b81f348394 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:39:51.070025 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:51.150775 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19606222-2c2e-41ee-b8ee-9a725d912356 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a64ed6c8e15d479887ab1dc59997d715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:39:51.151861 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19606222-2c2e-41ee-b8ee-9a725d912356 service nova] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:51.152211 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19606222-2c2e-41ee-b8ee-9a725d912356 service nova] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:51.152564 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19606222-2c2e-41ee-b8ee-9a725d912356 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:51.154847 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-19606222-2c2e-41ee-b8ee-9a725d912356 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:51.276066 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19606222-2c2e-41ee-b8ee-9a725d912356 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:39:51.278192 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19606222-2c2e-41ee-b8ee-9a725d912356 service nova] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:39:51.300449 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 688/1385] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:39:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:39:53.713146 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9028c4-0471-42c9-83db-4dc9ed73cb94 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:53.713679 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9028c4-0471-42c9-83db-4dc9ed73cb94 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:53.713882 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9028c4-0471-42c9-83db-4dc9ed73cb94 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:53.715934 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 698/1386] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:39:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:39:53.723268 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3232bd62-2d41-4a5b-9e22-2ce97cd19051 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:53.723800 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3232bd62-2d41-4a5b-9e22-2ce97cd19051 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:53.723997 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3232bd62-2d41-4a5b-9e22-2ce97cd19051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:53.837416 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-3232bd62-2d41-4a5b-9e22-2ce97cd19051 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:39:53.838669 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-3232bd62-2d41-4a5b-9e22-2ce97cd19051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:53.848333 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:53.977179 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 689/1387] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:39:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 02:39:54.292612 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9028c4-0471-42c9-83db-4dc9ed73cb94 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:39:54.294680 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-fe9028c4-0471-42c9-83db-4dc9ed73cb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:54.477462 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9028c4-0471-42c9-83db-4dc9ed73cb94 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:39:54.539107 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:54.641688 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a7f47e-113f-4bb6-aa62-6dc9a430afb3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a64ed6c8e15d479887ab1dc59997d715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:39:54.642448 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a7f47e-113f-4bb6-aa62-6dc9a430afb3 service nova] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:54.642738 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a7f47e-113f-4bb6-aa62-6dc9a430afb3 service nova] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:54.643039 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a7f47e-113f-4bb6-aa62-6dc9a430afb3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:54.644724 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-01a7f47e-113f-4bb6-aa62-6dc9a430afb3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:54.676553 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d3f59-3f34-4207-ab1c-6215d692fded None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:54.676927 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d3f59-3f34-4207-ab1c-6215d692fded None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:54.677395 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d3f59-3f34-4207-ab1c-6215d692fded None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:54.678957 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 690/1388] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:39:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:39:54.684469 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccace5cd-2e2e-46cf-bed3-f809c1bfdd34 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:54.685000 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccace5cd-2e2e-46cf-bed3-f809c1bfdd34 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:54.685278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccace5cd-2e2e-46cf-bed3-f809c1bfdd34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:54.757425 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-ccace5cd-2e2e-46cf-bed3-f809c1bfdd34 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:39:54.758653 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-ccace5cd-2e2e-46cf-bed3-f809c1bfdd34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:54.776817 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a7f47e-113f-4bb6-aa62-6dc9a430afb3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:39:54.778463 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a7f47e-113f-4bb6-aa62-6dc9a430afb3 service nova] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:39:54.798779 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 699/1389] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:39:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 02:39:54.873917 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:54.891920 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 691/1390] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:39:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 02:39:55.215219 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178e96f-9bb6-4c97-8963-272b2c4b5eaf None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:55.215602 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178e96f-9bb6-4c97-8963-272b2c4b5eaf None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:55.215949 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178e96f-9bb6-4c97-8963-272b2c4b5eaf None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:55.217405 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 700/1391] 10.222.0.20 () {54 vars in 962 bytes} [Thu Sep 22 02:39:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:39:55.224674 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3a3f5-ea9b-4f17-b616-72b94dbb31e3 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:55.225199 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3a3f5-ea9b-4f17-b616-72b94dbb31e3 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:55.225707 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3a3f5-ea9b-4f17-b616-72b94dbb31e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:55.333484 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-9ad3a3f5-ea9b-4f17-b616-72b94dbb31e3 None None] MFA Rules not processed for user `6160520562b74be3b25c49a30e990d8b`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:39:55.334740 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-9ad3a3f5-ea9b-4f17-b616-72b94dbb31e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:55.439276 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 692/1392] 10.222.0.20 () {58 vars in 1059 bytes} [Thu Sep 22 02:39:55 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:39:55.456417 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f178e96f-9bb6-4c97-8963-272b2c4b5eaf None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:39:55.458893 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-f178e96f-9bb6-4c97-8963-272b2c4b5eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:55.538379 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:55.606263 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f178e96f-9bb6-4c97-8963-272b2c4b5eaf None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:39:55.696123 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11008807-e786-4cf5-97ce-28b1c909a709 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160520562b74be3b25c49a30e990d8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:39:55.696648 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11008807-e786-4cf5-97ce-28b1c909a709 service glance] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:55.696927 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11008807-e786-4cf5-97ce-28b1c909a709 service glance] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:55.697197 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11008807-e786-4cf5-97ce-28b1c909a709 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:55.698868 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-11008807-e786-4cf5-97ce-28b1c909a709 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:55.821661 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:55.847452 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11008807-e786-4cf5-97ce-28b1c909a709 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:39:55.850306 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11008807-e786-4cf5-97ce-28b1c909a709 service glance] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:39:55.882336 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 701/1393] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:39:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:39:56.723117 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b960b-0795-4f11-9bd3-1556ad5737bb None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:56.724556 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b960b-0795-4f11-9bd3-1556ad5737bb None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:56.724556 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b960b-0795-4f11-9bd3-1556ad5737bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:56.810835 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-7a8b960b-0795-4f11-9bd3-1556ad5737bb None None] MFA Rules not processed for user `9821de2a961f4ddab2fca8e45b74e3db`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:39:56.812236 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-7a8b960b-0795-4f11-9bd3-1556ad5737bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:56.855656 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:56.934471 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 693/1394] 10.222.0.20 () {58 vars in 1009 bytes} [Thu Sep 22 02:39:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 02:39:56.987187 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ab3fa-60e6-4d58-85c6-caf2c4728eb7 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:56.987729 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ab3fa-60e6-4d58-85c6-caf2c4728eb7 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:56.987729 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ab3fa-60e6-4d58-85c6-caf2c4728eb7 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:56.989327 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 702/1395] 10.222.0.20 () {54 vars in 943 bytes} [Thu Sep 22 02:39:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:39:56.998650 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb540679-e757-4c18-8a73-2c4aeaf194d8 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:56.999103 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb540679-e757-4c18-8a73-2c4aeaf194d8 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:56.999467 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb540679-e757-4c18-8a73-2c4aeaf194d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:57.103724 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-cb540679-e757-4c18-8a73-2c4aeaf194d8 None None] MFA Rules not processed for user `4b9ad548622b4e54a6fecd0afd5ef3c2`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:39:57.104940 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-cb540679-e757-4c18-8a73-2c4aeaf194d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:57.196425 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3fa7d-2609-40bc-839e-5384a1a4e7de None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:57.196718 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3fa7d-2609-40bc-839e-5384a1a4e7de None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:57.196891 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3fa7d-2609-40bc-839e-5384a1a4e7de None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:57.198458 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 703/1396] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:39:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:39:57.207060 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c61a60-a606-4a09-b5dd-d110349dd9d0 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:57.207475 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c61a60-a606-4a09-b5dd-d110349dd9d0 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:57.207774 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c61a60-a606-4a09-b5dd-d110349dd9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:57.257830 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 694/1397] 10.222.0.20 () {58 vars in 1040 bytes} [Thu Sep 22 02:39:56 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:39:57.274138 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb540679-e757-4c18-8a73-2c4aeaf194d8 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:39:57.293447 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb540679-e757-4c18-8a73-2c4aeaf194d8 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:39:57.297053 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:57.325896 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-e1c61a60-a606-4a09-b5dd-d110349dd9d0 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:39:57.327398 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-e1c61a60-a606-4a09-b5dd-d110349dd9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:57.423287 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2100af2-d9ed-4d3b-937c-0d6484bb5278 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9ad548622b4e54a6fecd0afd5ef3c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:39:57.423917 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2100af2-d9ed-4d3b-937c-0d6484bb5278 service swift] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:57.424221 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2100af2-d9ed-4d3b-937c-0d6484bb5278 service swift] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:57.424487 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2100af2-d9ed-4d3b-937c-0d6484bb5278 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:57.426208 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-b2100af2-d9ed-4d3b-937c-0d6484bb5278 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:57.492032 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 704/1398] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:39:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 02:39:57.527997 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2100af2-d9ed-4d3b-937c-0d6484bb5278 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:39:57.529608 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2100af2-d9ed-4d3b-937c-0d6484bb5278 service swift] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:39:57.539106 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 695/1399] 10.222.0.20 () {60 vars in 1384 bytes} [Thu Sep 22 02:39:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 02:39:57.834189 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c61a60-a606-4a09-b5dd-d110349dd9d0 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:39:57.841387 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-e1c61a60-a606-4a09-b5dd-d110349dd9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:57.925058 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:57.991476 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c61a60-a606-4a09-b5dd-d110349dd9d0 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:39:58.169413 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a61df-015d-4d90-9c4b-95a97d48e578 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a64ed6c8e15d479887ab1dc59997d715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:39:58.170501 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a61df-015d-4d90-9c4b-95a97d48e578 service nova] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:39:58.171067 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a61df-015d-4d90-9c4b-95a97d48e578 service nova] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:39:58.171597 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a61df-015d-4d90-9c4b-95a97d48e578 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:39:58.177472 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-327a61df-015d-4d90-9c4b-95a97d48e578 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:39:58.266906 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:39:58.319011 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a61df-015d-4d90-9c4b-95a97d48e578 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:39:58.321094 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a61df-015d-4d90-9c4b-95a97d48e578 service nova] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:39:58.357304 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 705/1400] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:39:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 02:40:00.703665 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f7f3b-17fa-401d-aefe-7e8e712bae49 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:00.704588 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f7f3b-17fa-401d-aefe-7e8e712bae49 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:00.704588 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f7f3b-17fa-401d-aefe-7e8e712bae49 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:00.711928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57810ce3-fe95-4216-911a-b2061e564c97 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:00.712561 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57810ce3-fe95-4216-911a-b2061e564c97 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:00.712561 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57810ce3-fe95-4216-911a-b2061e564c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:00.717819 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 696/1401] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:40:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:40:00.788885 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-57810ce3-fe95-4216-911a-b2061e564c97 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:40:00.790127 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-57810ce3-fe95-4216-911a-b2061e564c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:00.923066 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 706/1402] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:40:01.207818 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f7f3b-17fa-401d-aefe-7e8e712bae49 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:40:01.209723 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-1c9f7f3b-17fa-401d-aefe-7e8e712bae49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:01.260995 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:01.478172 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f7f3b-17fa-401d-aefe-7e8e712bae49 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:40:01.627319 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:01.639263 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e4d55-95a2-482e-8b83-ef33a2e7a91c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a64ed6c8e15d479887ab1dc59997d715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:40:01.641164 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e4d55-95a2-482e-8b83-ef33a2e7a91c service nova] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:01.641476 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e4d55-95a2-482e-8b83-ef33a2e7a91c service nova] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:01.641753 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e4d55-95a2-482e-8b83-ef33a2e7a91c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:01.644746 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-368e4d55-95a2-482e-8b83-ef33a2e7a91c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:01.786687 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e4d55-95a2-482e-8b83-ef33a2e7a91c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:40:01.789359 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e4d55-95a2-482e-8b83-ef33a2e7a91c service nova] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:40:01.816114 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 697/1403] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:40:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 02:40:04.176384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18508229-75f7-40c8-b7ce-c382cba30530 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:04.177064 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18508229-75f7-40c8-b7ce-c382cba30530 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:04.177435 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18508229-75f7-40c8-b7ce-c382cba30530 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:04.179266 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 707/1404] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:40:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:40:04.186905 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fefd0df-6fdb-47b6-8c52-69181d2be67a None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:04.187752 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fefd0df-6fdb-47b6-8c52-69181d2be67a None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:04.187752 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fefd0df-6fdb-47b6-8c52-69181d2be67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:04.275078 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-0fefd0df-6fdb-47b6-8c52-69181d2be67a None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:40:04.276336 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-0fefd0df-6fdb-47b6-8c52-69181d2be67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:04.366394 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:04.419455 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 698/1405] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:40:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 02:40:04.735691 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18508229-75f7-40c8-b7ce-c382cba30530 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:40:04.738963 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-18508229-75f7-40c8-b7ce-c382cba30530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:04.798924 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:04.945821 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18508229-75f7-40c8-b7ce-c382cba30530 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:40:05.109409 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:05.118855 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc922e7-b878-4eda-9be9-624b17dd8b33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a64ed6c8e15d479887ab1dc59997d715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:40:05.119512 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc922e7-b878-4eda-9be9-624b17dd8b33 service nova] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:05.119987 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc922e7-b878-4eda-9be9-624b17dd8b33 service nova] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:05.120376 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc922e7-b878-4eda-9be9-624b17dd8b33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:05.122035 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-3dc922e7-b878-4eda-9be9-624b17dd8b33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:05.234901 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc922e7-b878-4eda-9be9-624b17dd8b33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:40:05.236513 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc922e7-b878-4eda-9be9-624b17dd8b33 service nova] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:40:05.262888 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 708/1406] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:40:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:40:07.608408 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10a65b-80d5-4924-93e2-ba944154d6ae None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:07.609014 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10a65b-80d5-4924-93e2-ba944154d6ae None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:07.609014 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10a65b-80d5-4924-93e2-ba944154d6ae None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:07.610585 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 699/1407] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:40:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:40:07.616083 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017de75e-802d-4d10-9532-e3a82ecb9cef None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:07.616820 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017de75e-802d-4d10-9532-e3a82ecb9cef None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:07.616820 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017de75e-802d-4d10-9532-e3a82ecb9cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:07.708739 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-017de75e-802d-4d10-9532-e3a82ecb9cef None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:40:07.708739 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-017de75e-802d-4d10-9532-e3a82ecb9cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:07.790522 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:07.851036 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 709/1408] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:40:08.128041 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c10a65b-80d5-4924-93e2-ba944154d6ae None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:40:08.129882 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-7c10a65b-80d5-4924-93e2-ba944154d6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:08.295591 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c10a65b-80d5-4924-93e2-ba944154d6ae None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:40:08.352861 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:08.425013 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450269c-8d20-494a-a6a8-44e63a76cd46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a64ed6c8e15d479887ab1dc59997d715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:40:08.425634 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450269c-8d20-494a-a6a8-44e63a76cd46 service nova] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:08.425921 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450269c-8d20-494a-a6a8-44e63a76cd46 service nova] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:08.426267 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450269c-8d20-494a-a6a8-44e63a76cd46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:08.428510 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-b450269c-8d20-494a-a6a8-44e63a76cd46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:08.539981 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b450269c-8d20-494a-a6a8-44e63a76cd46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:40:08.541664 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b450269c-8d20-494a-a6a8-44e63a76cd46 service nova] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:40:08.563541 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 700/1409] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:40:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:40:17.831527 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=13 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:18.559674 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=74 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:21.416536 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9729f-a813-45c8-8a65-acd0f2402180 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:21.417117 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9729f-a813-45c8-8a65-acd0f2402180 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:21.417117 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9729f-a813-45c8-8a65-acd0f2402180 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:21.418449 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 710/1410] 10.222.0.20 () {54 vars in 918 bytes} [Thu Sep 22 02:40:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:40:21.423718 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e5a18-f300-4011-84e8-361e132dec40 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:21.424184 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e5a18-f300-4011-84e8-361e132dec40 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:21.424277 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e5a18-f300-4011-84e8-361e132dec40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:21.502529 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-cb1e5a18-f300-4011-84e8-361e132dec40 None None] MFA Rules not processed for user `865576e9e17d4efdbd1ca6a005f31bda`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:40:21.503794 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-cb1e5a18-f300-4011-84e8-361e132dec40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:21.611159 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 701/1411] 10.222.0.20 () {58 vars in 1015 bytes} [Thu Sep 22 02:40:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:40:21.626253 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff934ff-accd-4931-94c3-2aaa97044235 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:21.626545 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff934ff-accd-4931-94c3-2aaa97044235 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:21.626849 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff934ff-accd-4931-94c3-2aaa97044235 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:21.628198 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 711/1412] 10.222.0.20 () {54 vars in 918 bytes} [Thu Sep 22 02:40:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:40:21.634026 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a81f3c-9c02-4937-8579-835747c914e3 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:21.634224 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a81f3c-9c02-4937-8579-835747c914e3 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:21.634508 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a81f3c-9c02-4937-8579-835747c914e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:21.716519 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:21.719287 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-29a81f3c-9c02-4937-8579-835747c914e3 None None] MFA Rules not processed for user `a64ed6c8e15d479887ab1dc59997d715`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:40:21.721121 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-29a81f3c-9c02-4937-8579-835747c914e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:21.838905 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 702/1413] 10.222.0.20 () {58 vars in 1015 bytes} [Thu Sep 22 02:40:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:40:24.456955 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda69880-d9f3-4cf7-9079-9d715838fef3 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:24.457540 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda69880-d9f3-4cf7-9079-9d715838fef3 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:24.457540 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda69880-d9f3-4cf7-9079-9d715838fef3 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:24.458904 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 712/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 02:40:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:40:24.464368 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b257d3d-5307-4512-b6a4-c5665c0925c1 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:24.464821 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b257d3d-5307-4512-b6a4-c5665c0925c1 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:24.464923 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b257d3d-5307-4512-b6a4-c5665c0925c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:24.538649 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-0b257d3d-5307-4512-b6a4-c5665c0925c1 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:40:24.539911 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-0b257d3d-5307-4512-b6a4-c5665c0925c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:24.599829 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:24.644608 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 703/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 02:40:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:40:24.944269 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda69880-d9f3-4cf7-9079-9d715838fef3 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:40:24.945831 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-fda69880-d9f3-4cf7-9079-9d715838fef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:25.104104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda69880-d9f3-4cf7-9079-9d715838fef3 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:40:25.180116 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:25.221963 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10102f2e-1eab-4bcd-aa19-3f96e130a367 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a64ed6c8e15d479887ab1dc59997d715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:40:25.222573 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10102f2e-1eab-4bcd-aa19-3f96e130a367 service nova] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:25.222879 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10102f2e-1eab-4bcd-aa19-3f96e130a367 service nova] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:25.223155 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10102f2e-1eab-4bcd-aa19-3f96e130a367 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:25.225452 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-10102f2e-1eab-4bcd-aa19-3f96e130a367 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:25.314601 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10102f2e-1eab-4bcd-aa19-3f96e130a367 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:40:25.316254 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10102f2e-1eab-4bcd-aa19-3f96e130a367 service nova] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:40:25.334900 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 713/1416] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:40:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:40:26.494226 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd207e-5fca-4b59-91bb-db42126f5022 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:26.494876 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd207e-5fca-4b59-91bb-db42126f5022 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:26.494876 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd207e-5fca-4b59-91bb-db42126f5022 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:26.496140 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 704/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 02:40:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:40:26.502844 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06759d-f1f3-4927-9d9b-26d4a1926d06 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:26.503426 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06759d-f1f3-4927-9d9b-26d4a1926d06 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:26.503426 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06759d-f1f3-4927-9d9b-26d4a1926d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:26.583080 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-dc06759d-f1f3-4927-9d9b-26d4a1926d06 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:40:26.584338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-dc06759d-f1f3-4927-9d9b-26d4a1926d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:26.614752 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:26.691449 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 714/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 02:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:40:26.846713 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd207e-5fca-4b59-91bb-db42126f5022 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:40:26.848427 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-7afd207e-5fca-4b59-91bb-db42126f5022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:27.008032 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd207e-5fca-4b59-91bb-db42126f5022 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:40:27.049972 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:27.123525 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553d965-e30d-4177-b3fc-b3e8c268a61c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a64ed6c8e15d479887ab1dc59997d715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:40:27.124025 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553d965-e30d-4177-b3fc-b3e8c268a61c service nova] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:27.124357 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553d965-e30d-4177-b3fc-b3e8c268a61c service nova] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:27.124583 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553d965-e30d-4177-b3fc-b3e8c268a61c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:27.126198 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-5553d965-e30d-4177-b3fc-b3e8c268a61c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:27.214857 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5553d965-e30d-4177-b3fc-b3e8c268a61c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:40:27.216497 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5553d965-e30d-4177-b3fc-b3e8c268a61c service nova] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:40:27.234900 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 705/1419] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:40:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:40:28.384438 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603eaeb3-47ff-4c4b-a89b-fe11538f5c2a None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:28.385026 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603eaeb3-47ff-4c4b-a89b-fe11538f5c2a None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:28.385026 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603eaeb3-47ff-4c4b-a89b-fe11538f5c2a None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:28.386352 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 715/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 02:40:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:40:28.393040 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c225d-3a58-451f-aac9-4f26e2f7b33e None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:28.393533 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c225d-3a58-451f-aac9-4f26e2f7b33e None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:28.393639 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c225d-3a58-451f-aac9-4f26e2f7b33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:28.470353 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:28.495140 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-0a6c225d-3a58-451f-aac9-4f26e2f7b33e None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:40:28.496374 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-0a6c225d-3a58-451f-aac9-4f26e2f7b33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:28.603197 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 706/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 02:40:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:40:28.762531 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603eaeb3-47ff-4c4b-a89b-fe11538f5c2a None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:40:28.764068 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-603eaeb3-47ff-4c4b-a89b-fe11538f5c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:28.911921 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603eaeb3-47ff-4c4b-a89b-fe11538f5c2a None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:40:28.924136 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:29.040284 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc5393-ea41-4f87-aacb-681fbab71fe0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a64ed6c8e15d479887ab1dc59997d715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:40:29.040903 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc5393-ea41-4f87-aacb-681fbab71fe0 service nova] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:29.041185 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc5393-ea41-4f87-aacb-681fbab71fe0 service nova] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:29.041466 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc5393-ea41-4f87-aacb-681fbab71fe0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:29.043129 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-8cdc5393-ea41-4f87-aacb-681fbab71fe0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:29.135123 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc5393-ea41-4f87-aacb-681fbab71fe0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:40:29.136735 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc5393-ea41-4f87-aacb-681fbab71fe0 service nova] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:40:29.156037 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 716/1422] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:40:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:40:30.293614 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b018fcf-d939-4ea6-b8cd-be9d22c05945 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:30.294161 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b018fcf-d939-4ea6-b8cd-be9d22c05945 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:30.294161 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b018fcf-d939-4ea6-b8cd-be9d22c05945 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:30.295588 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 707/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 02:40:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:40:30.303773 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f74868-45c0-4fcb-82f1-69a297511668 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:30.304388 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f74868-45c0-4fcb-82f1-69a297511668 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:30.304388 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f74868-45c0-4fcb-82f1-69a297511668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:30.325464 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:30.386409 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-a8f74868-45c0-4fcb-82f1-69a297511668 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:40:30.387664 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-a8f74868-45c0-4fcb-82f1-69a297511668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:30.498135 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 717/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 02:40:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:40:30.653847 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b018fcf-d939-4ea6-b8cd-be9d22c05945 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:40:30.655440 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-5b018fcf-d939-4ea6-b8cd-be9d22c05945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:30.777550 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:30.815338 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b018fcf-d939-4ea6-b8cd-be9d22c05945 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:40:30.930246 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d175d59-eaae-47cf-acb8-96a0fcc5fdbb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a64ed6c8e15d479887ab1dc59997d715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:40:30.930768 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d175d59-eaae-47cf-acb8-96a0fcc5fdbb service nova] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:30.931077 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d175d59-eaae-47cf-acb8-96a0fcc5fdbb service nova] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:30.931356 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d175d59-eaae-47cf-acb8-96a0fcc5fdbb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:30.932957 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-1d175d59-eaae-47cf-acb8-96a0fcc5fdbb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:31.009140 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:31.024582 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d175d59-eaae-47cf-acb8-96a0fcc5fdbb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:40:31.026290 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d175d59-eaae-47cf-acb8-96a0fcc5fdbb service nova] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:40:31.045151 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 708/1425] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:40:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:40:32.190857 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e23898-189e-47a5-93da-0591d3d99b05 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:32.191416 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e23898-189e-47a5-93da-0591d3d99b05 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:32.191416 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e23898-189e-47a5-93da-0591d3d99b05 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:32.192734 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 718/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 02:40:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:40:32.199491 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c3b67-b62e-4606-93c7-1518e9d95a98 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:32.200028 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c3b67-b62e-4606-93c7-1518e9d95a98 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:32.200028 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c3b67-b62e-4606-93c7-1518e9d95a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:32.289267 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-119c3b67-b62e-4606-93c7-1518e9d95a98 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:40:32.290553 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-119c3b67-b62e-4606-93c7-1518e9d95a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:32.391784 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 709/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 02:40:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 02:40:32.545557 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e23898-189e-47a5-93da-0591d3d99b05 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:40:32.547041 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-c6e23898-189e-47a5-93da-0591d3d99b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:32.641591 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:32.726301 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e23898-189e-47a5-93da-0591d3d99b05 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:40:32.841725 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bde0e75-d3a8-4590-81b4-a54feaf95871 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a64ed6c8e15d479887ab1dc59997d715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:40:32.842330 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde0e75-d3a8-4590-81b4-a54feaf95871 service nova] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:32.842609 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde0e75-d3a8-4590-81b4-a54feaf95871 service nova] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:32.842944 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde0e75-d3a8-4590-81b4-a54feaf95871 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:32.845168 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-8bde0e75-d3a8-4590-81b4-a54feaf95871 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:32.894111 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:32.935697 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bde0e75-d3a8-4590-81b4-a54feaf95871 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:40:32.937328 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bde0e75-d3a8-4590-81b4-a54feaf95871 service nova] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:40:32.955907 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 719/1428] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:40:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:40:34.036802 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59512b8-17b4-42f3-9359-68c6e53fe703 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:34.037323 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59512b8-17b4-42f3-9359-68c6e53fe703 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:34.037424 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59512b8-17b4-42f3-9359-68c6e53fe703 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:34.038718 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 710/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 02:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:40:34.045449 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b092e-dee4-4e5e-a602-93d49bf30ce0 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:34.045979 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b092e-dee4-4e5e-a602-93d49bf30ce0 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:34.045979 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b092e-dee4-4e5e-a602-93d49bf30ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:34.129811 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-605b092e-dee4-4e5e-a602-93d49bf30ce0 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:40:34.131735 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-605b092e-dee4-4e5e-a602-93d49bf30ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:34.243250 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 720/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 02:40:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:40:34.392150 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59512b8-17b4-42f3-9359-68c6e53fe703 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:40:34.394121 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-c59512b8-17b4-42f3-9359-68c6e53fe703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:34.445584 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:34.548449 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59512b8-17b4-42f3-9359-68c6e53fe703 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:40:34.665481 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf17d3cd-1ece-43c6-8d21-37c28456e9c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a64ed6c8e15d479887ab1dc59997d715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:40:34.666007 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17d3cd-1ece-43c6-8d21-37c28456e9c8 service nova] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:34.666292 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17d3cd-1ece-43c6-8d21-37c28456e9c8 service nova] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:34.666569 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17d3cd-1ece-43c6-8d21-37c28456e9c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:34.668248 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-cf17d3cd-1ece-43c6-8d21-37c28456e9c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:34.675440 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:34.763202 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf17d3cd-1ece-43c6-8d21-37c28456e9c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:40:34.764846 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf17d3cd-1ece-43c6-8d21-37c28456e9c8 service nova] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:40:34.784764 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 711/1431] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:40:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:40:35.849797 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc3567-6334-4364-82df-22e5fb65bd91 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:35.850370 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc3567-6334-4364-82df-22e5fb65bd91 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:35.850370 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc3567-6334-4364-82df-22e5fb65bd91 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:35.851757 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 721/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 02:40:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:40:35.858799 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e16df-4113-4543-97f5-6fa3410e3559 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:35.859210 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e16df-4113-4543-97f5-6fa3410e3559 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:35.859210 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e16df-4113-4543-97f5-6fa3410e3559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:35.936421 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-fd2e16df-4113-4543-97f5-6fa3410e3559 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:40:35.937654 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-fd2e16df-4113-4543-97f5-6fa3410e3559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:36.008219 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:36.044853 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 712/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 02:40:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:40:36.197039 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cc3567-6334-4364-82df-22e5fb65bd91 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:40:36.198481 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-17cc3567-6334-4364-82df-22e5fb65bd91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:36.216934 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:36.355594 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cc3567-6334-4364-82df-22e5fb65bd91 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:40:36.444472 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:36.471876 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea99b7-f428-48a3-a039-d3473b4e1861 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a64ed6c8e15d479887ab1dc59997d715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:40:36.472413 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea99b7-f428-48a3-a039-d3473b4e1861 service nova] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:36.472698 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea99b7-f428-48a3-a039-d3473b4e1861 service nova] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:36.472976 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea99b7-f428-48a3-a039-d3473b4e1861 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:36.475293 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-67ea99b7-f428-48a3-a039-d3473b4e1861 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:36.567028 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ea99b7-f428-48a3-a039-d3473b4e1861 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:40:36.568675 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ea99b7-f428-48a3-a039-d3473b4e1861 service nova] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:40:36.587274 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 722/1434] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:40:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:40:37.666896 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b7124-7173-4f61-84cc-709cf820c51a None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:37.667465 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b7124-7173-4f61-84cc-709cf820c51a None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:37.667465 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b7124-7173-4f61-84cc-709cf820c51a None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:37.668786 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 713/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 02:40:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:40:37.675397 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614336c-8d72-4415-8b8c-db58ca8a761a None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:37.675928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614336c-8d72-4415-8b8c-db58ca8a761a None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:37.675928 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614336c-8d72-4415-8b8c-db58ca8a761a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:37.753293 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-c614336c-8d72-4415-8b8c-db58ca8a761a None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:40:37.754437 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-c614336c-8d72-4415-8b8c-db58ca8a761a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:37.797212 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:37.852287 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 723/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 02:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:40:38.001278 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b7124-7173-4f61-84cc-709cf820c51a None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:40:38.002863 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-7a4b7124-7173-4f61-84cc-709cf820c51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:38.152439 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b7124-7173-4f61-84cc-709cf820c51a None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:40:38.206432 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:38.265696 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d2cbf0-a09e-49d5-9e63-160591169324 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a64ed6c8e15d479887ab1dc59997d715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:40:38.266309 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2cbf0-a09e-49d5-9e63-160591169324 service nova] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:38.266595 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2cbf0-a09e-49d5-9e63-160591169324 service nova] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:38.266950 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2cbf0-a09e-49d5-9e63-160591169324 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:38.268858 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-c2d2cbf0-a09e-49d5-9e63-160591169324 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:38.358809 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d2cbf0-a09e-49d5-9e63-160591169324 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:40:38.360460 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d2cbf0-a09e-49d5-9e63-160591169324 service nova] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:40:38.379513 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 714/1437] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:40:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:40:39.464050 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001a93d-6006-4b51-bdf4-3be9f08a1e52 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:39.464552 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001a93d-6006-4b51-bdf4-3be9f08a1e52 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:39.464664 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001a93d-6006-4b51-bdf4-3be9f08a1e52 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:39.465976 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 724/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 02:40:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:40:39.472638 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974454bb-5821-4668-b157-df036110cb2f None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:39.473150 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974454bb-5821-4668-b157-df036110cb2f None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:39.473271 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974454bb-5821-4668-b157-df036110cb2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:39.556248 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-974454bb-5821-4668-b157-df036110cb2f None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:40:39.557495 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-974454bb-5821-4668-b157-df036110cb2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:39.559679 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:39.658966 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 715/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 02:40:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:40:39.810379 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6001a93d-6006-4b51-bdf4-3be9f08a1e52 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:40:39.811848 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-6001a93d-6006-4b51-bdf4-3be9f08a1e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:39.963566 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6001a93d-6006-4b51-bdf4-3be9f08a1e52 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:40:39.993068 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:40.085555 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292f36f-473c-4ae3-8da8-88d914e2b090 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a64ed6c8e15d479887ab1dc59997d715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:40:40.086255 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292f36f-473c-4ae3-8da8-88d914e2b090 service nova] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:40.086539 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292f36f-473c-4ae3-8da8-88d914e2b090 service nova] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:40.086813 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292f36f-473c-4ae3-8da8-88d914e2b090 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:40.088499 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-4292f36f-473c-4ae3-8da8-88d914e2b090 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:40.180756 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292f36f-473c-4ae3-8da8-88d914e2b090 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:40:40.182384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292f36f-473c-4ae3-8da8-88d914e2b090 service nova] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:40:40.201493 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 725/1440] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:40:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:40:42.362567 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85bdfde-f87d-44c3-af69-e9543928a0fe None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:42.363359 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85bdfde-f87d-44c3-af69-e9543928a0fe None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:42.363359 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85bdfde-f87d-44c3-af69-e9543928a0fe None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:42.365032 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 716/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 02:40:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 02:40:42.370971 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6eea8-f51c-44cf-82eb-7ca853453208 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:42.371423 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6eea8-f51c-44cf-82eb-7ca853453208 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:42.371551 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6eea8-f51c-44cf-82eb-7ca853453208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:42.412846 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:42.440212 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-d3a6eea8-f51c-44cf-82eb-7ca853453208 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:40:42.441445 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-d3a6eea8-f51c-44cf-82eb-7ca853453208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:42.544018 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 726/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 02:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:40:42.839185 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85bdfde-f87d-44c3-af69-e9543928a0fe None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:40:42.840763 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-f85bdfde-f87d-44c3-af69-e9543928a0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:42.948882 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85bdfde-f87d-44c3-af69-e9543928a0fe None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:40:42.981793 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:43.024648 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f292756-d018-4aeb-ba5f-722f5c721c5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f635200ad241b3b31ed49701bc511b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:40:43.025408 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f292756-d018-4aeb-ba5f-722f5c721c5e service neutron] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:43.025542 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f292756-d018-4aeb-ba5f-722f5c721c5e service neutron] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:43.025823 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f292756-d018-4aeb-ba5f-722f5c721c5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:43.027491 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-1f292756-d018-4aeb-ba5f-722f5c721c5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:43.138048 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f292756-d018-4aeb-ba5f-722f5c721c5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:40:43.139453 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f292756-d018-4aeb-ba5f-722f5c721c5e service neutron] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:40:43.158003 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 717/1443] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:40:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:40:45.534095 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c14438-f6ea-4de3-a2dd-f62b58d160f8 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:45.534673 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c14438-f6ea-4de3-a2dd-f62b58d160f8 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:45.534673 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c14438-f6ea-4de3-a2dd-f62b58d160f8 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:45.536013 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 727/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 02:40:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:40:45.541744 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307edd7-56e1-4b3b-b248-f073c6403699 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:45.542375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307edd7-56e1-4b3b-b248-f073c6403699 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:45.542375 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307edd7-56e1-4b3b-b248-f073c6403699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:45.609616 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-f307edd7-56e1-4b3b-b248-f073c6403699 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:40:45.610857 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-f307edd7-56e1-4b3b-b248-f073c6403699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:45.624456 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:45.715573 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 718/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 02:40:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:40:46.007875 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c14438-f6ea-4de3-a2dd-f62b58d160f8 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:40:46.009459 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-b0c14438-f6ea-4de3-a2dd-f62b58d160f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:46.115747 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c14438-f6ea-4de3-a2dd-f62b58d160f8 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:40:46.128659 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:46.190765 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b458a353-deff-4d63-9ac2-7e210296c80b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f635200ad241b3b31ed49701bc511b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:40:46.191408 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458a353-deff-4d63-9ac2-7e210296c80b service neutron] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:46.191692 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458a353-deff-4d63-9ac2-7e210296c80b service neutron] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:46.191965 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458a353-deff-4d63-9ac2-7e210296c80b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:46.193609 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-b458a353-deff-4d63-9ac2-7e210296c80b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:46.426853 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b458a353-deff-4d63-9ac2-7e210296c80b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:40:46.428686 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b458a353-deff-4d63-9ac2-7e210296c80b service neutron] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:40:46.453537 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 728/1446] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:40:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:40:49.000226 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4e622-5ffe-4fde-9f40-a7ff4357a1e5 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:49.000783 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4e622-5ffe-4fde-9f40-a7ff4357a1e5 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:49.000783 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4e622-5ffe-4fde-9f40-a7ff4357a1e5 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:49.002132 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 719/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 02:40:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:40:49.008201 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5db08a-93de-43e7-ac97-23f9c74e0963 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:49.008809 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5db08a-93de-43e7-ac97-23f9c74e0963 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:49.008809 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5db08a-93de-43e7-ac97-23f9c74e0963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:49.060225 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:49.078917 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-4b5db08a-93de-43e7-ac97-23f9c74e0963 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:40:49.080136 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-4b5db08a-93de-43e7-ac97-23f9c74e0963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:49.179008 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 729/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 02:40:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:40:49.463703 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4e622-5ffe-4fde-9f40-a7ff4357a1e5 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:40:49.465465 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-b1c4e622-5ffe-4fde-9f40-a7ff4357a1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:49.575997 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4e622-5ffe-4fde-9f40-a7ff4357a1e5 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:40:49.620485 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:49.661991 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d418ad8b-b4e2-45b3-8124-d617e9a3fa8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f635200ad241b3b31ed49701bc511b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:40:49.662615 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418ad8b-b4e2-45b3-8124-d617e9a3fa8e service neutron] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:49.662928 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418ad8b-b4e2-45b3-8124-d617e9a3fa8e service neutron] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:49.663204 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418ad8b-b4e2-45b3-8124-d617e9a3fa8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:49.664858 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-d418ad8b-b4e2-45b3-8124-d617e9a3fa8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:49.777899 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d418ad8b-b4e2-45b3-8124-d617e9a3fa8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:40:49.779307 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d418ad8b-b4e2-45b3-8124-d617e9a3fa8e service neutron] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:40:49.798694 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 720/1449] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:40:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:40:52.237978 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01495527-85be-46a9-a8be-6ffdf12414e0 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:52.238547 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01495527-85be-46a9-a8be-6ffdf12414e0 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:52.238547 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01495527-85be-46a9-a8be-6ffdf12414e0 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:52.239933 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 730/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 02:40:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:40:52.245432 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee24459-d943-4214-9ae8-24c4ab7a2f2a None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:52.245878 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee24459-d943-4214-9ae8-24c4ab7a2f2a None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:52.245975 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee24459-d943-4214-9ae8-24c4ab7a2f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:52.313578 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-2ee24459-d943-4214-9ae8-24c4ab7a2f2a None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:40:52.314748 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-2ee24459-d943-4214-9ae8-24c4ab7a2f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:52.335801 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:52.413710 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 721/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 02:40:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:40:52.700351 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01495527-85be-46a9-a8be-6ffdf12414e0 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:40:52.701883 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-01495527-85be-46a9-a8be-6ffdf12414e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:52.809507 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01495527-85be-46a9-a8be-6ffdf12414e0 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:40:52.830378 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:52.898481 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df1c14-c550-4d7b-9ac7-c54151ef6b14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f635200ad241b3b31ed49701bc511b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:40:52.899201 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df1c14-c550-4d7b-9ac7-c54151ef6b14 service neutron] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:52.899457 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df1c14-c550-4d7b-9ac7-c54151ef6b14 service neutron] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:52.899739 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df1c14-c550-4d7b-9ac7-c54151ef6b14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:52.901551 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-56df1c14-c550-4d7b-9ac7-c54151ef6b14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:53.027547 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df1c14-c550-4d7b-9ac7-c54151ef6b14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:40:53.029006 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df1c14-c550-4d7b-9ac7-c54151ef6b14 service neutron] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:40:53.048721 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 731/1452] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:40:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:40:55.325703 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca94f5-c852-4795-9484-20353bb896a0 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:55.326279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca94f5-c852-4795-9484-20353bb896a0 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:55.326279 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca94f5-c852-4795-9484-20353bb896a0 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:55.327694 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 722/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 02:40:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:40:55.333678 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2c0ac-7006-4966-946f-acb87c96d1ca None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:55.334105 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2c0ac-7006-4966-946f-acb87c96d1ca None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:55.334201 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2c0ac-7006-4966-946f-acb87c96d1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:55.398592 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:55.401608 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-c8b2c0ac-7006-4966-946f-acb87c96d1ca None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:40:55.403434 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-c8b2c0ac-7006-4966-946f-acb87c96d1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:55.499998 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 732/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 02:40:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:40:55.785730 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca94f5-c852-4795-9484-20353bb896a0 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:40:55.787364 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-a0ca94f5-c852-4795-9484-20353bb896a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:55.899489 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca94f5-c852-4795-9484-20353bb896a0 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:40:56.077496 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:56.104161 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b746854-65a0-4457-87f9-40b91748310d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f635200ad241b3b31ed49701bc511b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:40:56.104771 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b746854-65a0-4457-87f9-40b91748310d service neutron] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:56.105053 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b746854-65a0-4457-87f9-40b91748310d service neutron] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:56.105327 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b746854-65a0-4457-87f9-40b91748310d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:56.106987 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-2b746854-65a0-4457-87f9-40b91748310d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:56.215870 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b746854-65a0-4457-87f9-40b91748310d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:40:56.217350 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b746854-65a0-4457-87f9-40b91748310d service neutron] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:40:56.235599 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 723/1455] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:40:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:40:58.538846 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c866dc37-d793-4acd-a7e5-5d467ad675dc None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:58.539492 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c866dc37-d793-4acd-a7e5-5d467ad675dc None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:58.539492 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c866dc37-d793-4acd-a7e5-5d467ad675dc None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:58.540741 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 733/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 02:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:40:58.546155 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47250f-af12-468f-aa48-f1709465064f None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:58.546655 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47250f-af12-468f-aa48-f1709465064f None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:58.546766 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47250f-af12-468f-aa48-f1709465064f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:58.617641 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-fe47250f-af12-468f-aa48-f1709465064f None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:40:58.618904 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-fe47250f-af12-468f-aa48-f1709465064f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:58.650429 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:58.720354 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 724/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 02:40:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:40:59.019523 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c866dc37-d793-4acd-a7e5-5d467ad675dc None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:40:59.021665 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-c866dc37-d793-4acd-a7e5-5d467ad675dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:59.133442 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c866dc37-d793-4acd-a7e5-5d467ad675dc None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:40:59.164637 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:40:59.211697 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be5842a-5043-4390-b2a1-14471f302378 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f635200ad241b3b31ed49701bc511b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:40:59.212312 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5842a-5043-4390-b2a1-14471f302378 service neutron] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:40:59.212609 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5842a-5043-4390-b2a1-14471f302378 service neutron] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:40:59.212870 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5842a-5043-4390-b2a1-14471f302378 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:40:59.214498 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-7be5842a-5043-4390-b2a1-14471f302378 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:40:59.326588 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be5842a-5043-4390-b2a1-14471f302378 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:40:59.328075 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be5842a-5043-4390-b2a1-14471f302378 service neutron] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:40:59.347150 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 734/1458] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:40:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:41:01.602252 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c56fd-36ef-474c-8783-d598b96c9e59 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:41:01.603580 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c56fd-36ef-474c-8783-d598b96c9e59 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:41:01.603580 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c56fd-36ef-474c-8783-d598b96c9e59 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:41:01.604198 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 725/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 02:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:41:01.609695 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4488c6-85eb-4567-b8e6-68f326faf852 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:41:01.610135 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4488c6-85eb-4567-b8e6-68f326faf852 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:41:01.610251 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4488c6-85eb-4567-b8e6-68f326faf852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:41:01.678626 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-2a4488c6-85eb-4567-b8e6-68f326faf852 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:41:01.679914 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-2a4488c6-85eb-4567-b8e6-68f326faf852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:41:01.689370 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:41:01.780148 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 735/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 02:41:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:41:02.068778 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c56fd-36ef-474c-8783-d598b96c9e59 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:41:02.070359 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-6b8c56fd-36ef-474c-8783-d598b96c9e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:41:02.177245 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c56fd-36ef-474c-8783-d598b96c9e59 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:41:02.241942 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:41:02.263486 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d6011-2113-4f48-a7e6-42441d31fc5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f635200ad241b3b31ed49701bc511b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:41:02.264116 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d6011-2113-4f48-a7e6-42441d31fc5f service neutron] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:41:02.264396 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d6011-2113-4f48-a7e6-42441d31fc5f service neutron] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:41:02.264845 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d6011-2113-4f48-a7e6-42441d31fc5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:41:02.268672 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-9f1d6011-2113-4f48-a7e6-42441d31fc5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:41:02.404998 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1d6011-2113-4f48-a7e6-42441d31fc5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:41:02.406450 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1d6011-2113-4f48-a7e6-42441d31fc5f service neutron] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:41:02.425517 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 726/1461] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:41:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:41:04.697599 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daccf3c9-a99a-4943-9c8b-2244fe079580 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:41:04.698154 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daccf3c9-a99a-4943-9c8b-2244fe079580 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:41:04.698154 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daccf3c9-a99a-4943-9c8b-2244fe079580 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:41:04.699519 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 736/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 02:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:41:04.704842 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb5a81-e76d-4db3-8e04-fed54a7e1eb6 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:41:04.705274 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb5a81-e76d-4db3-8e04-fed54a7e1eb6 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:41:04.705386 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb5a81-e76d-4db3-8e04-fed54a7e1eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:41:04.770726 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-afeb5a81-e76d-4db3-8e04-fed54a7e1eb6 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:41:04.772579 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-afeb5a81-e76d-4db3-8e04-fed54a7e1eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:41:04.810970 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:41:04.872825 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 727/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 02:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:41:05.160042 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daccf3c9-a99a-4943-9c8b-2244fe079580 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:41:05.161646 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-daccf3c9-a99a-4943-9c8b-2244fe079580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:41:05.278530 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daccf3c9-a99a-4943-9c8b-2244fe079580 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:41:05.325192 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:41:05.364158 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c7159-c45f-4df4-a3b7-779b3210c4cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f635200ad241b3b31ed49701bc511b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:41:05.365384 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c7159-c45f-4df4-a3b7-779b3210c4cf service neutron] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:41:05.365667 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c7159-c45f-4df4-a3b7-779b3210c4cf service neutron] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:41:05.365942 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c7159-c45f-4df4-a3b7-779b3210c4cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:41:05.367587 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-109c7159-c45f-4df4-a3b7-779b3210c4cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:41:05.482104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109c7159-c45f-4df4-a3b7-779b3210c4cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:41:05.483588 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109c7159-c45f-4df4-a3b7-779b3210c4cf service neutron] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:41:05.502278 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 737/1464] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:41:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:41:07.746914 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3f757-f55e-48c4-9ff7-7b5839f70344 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:41:07.747482 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3f757-f55e-48c4-9ff7-7b5839f70344 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:41:07.747482 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3f757-f55e-48c4-9ff7-7b5839f70344 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:41:07.748875 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 728/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 02:41:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:41:07.754331 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe83344-819b-4f15-98a8-94e558d899b4 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:41:07.754789 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe83344-819b-4f15-98a8-94e558d899b4 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:41:07.754920 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe83344-819b-4f15-98a8-94e558d899b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:41:07.823636 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-ffe83344-819b-4f15-98a8-94e558d899b4 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:41:07.826167 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-ffe83344-819b-4f15-98a8-94e558d899b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:41:07.845725 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:41:07.930644 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 738/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 02:41:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:41:08.230033 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef3f757-f55e-48c4-9ff7-7b5839f70344 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:41:08.231617 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-eef3f757-f55e-48c4-9ff7-7b5839f70344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:41:08.346767 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef3f757-f55e-48c4-9ff7-7b5839f70344 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:41:08.424328 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:41:08.432992 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c37e2f-fc58-4fb7-a993-2af9aa1a9d52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f635200ad241b3b31ed49701bc511b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:41:08.433614 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c37e2f-fc58-4fb7-a993-2af9aa1a9d52 service neutron] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:41:08.433892 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c37e2f-fc58-4fb7-a993-2af9aa1a9d52 service neutron] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:41:08.434162 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c37e2f-fc58-4fb7-a993-2af9aa1a9d52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:41:08.435892 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-95c37e2f-fc58-4fb7-a993-2af9aa1a9d52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:41:08.561954 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c37e2f-fc58-4fb7-a993-2af9aa1a9d52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:41:08.563466 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c37e2f-fc58-4fb7-a993-2af9aa1a9d52 service neutron] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:41:08.582332 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 729/1467] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:41:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:41:10.783109 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c39e42b-b621-481e-a48e-caee9c341e6f None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:41:10.783707 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c39e42b-b621-481e-a48e-caee9c341e6f None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:41:10.783707 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c39e42b-b621-481e-a48e-caee9c341e6f None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:41:10.785010 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 739/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 02:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:41:10.790373 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789a80b-341e-4b22-a53e-5743a3628dd9 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:41:10.790839 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789a80b-341e-4b22-a53e-5743a3628dd9 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:41:10.790957 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789a80b-341e-4b22-a53e-5743a3628dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:41:10.859825 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-f789a80b-341e-4b22-a53e-5743a3628dd9 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:41:10.861074 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-f789a80b-341e-4b22-a53e-5743a3628dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:41:10.911594 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:41:10.963740 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 730/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 02:41:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:41:11.248117 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c39e42b-b621-481e-a48e-caee9c341e6f None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:41:11.249724 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-7c39e42b-b621-481e-a48e-caee9c341e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:41:11.355922 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c39e42b-b621-481e-a48e-caee9c341e6f None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:41:11.406543 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:41:11.432835 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e373f-28f9-40e9-aa76-2340ec2395ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f635200ad241b3b31ed49701bc511b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:41:11.433448 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e373f-28f9-40e9-aa76-2340ec2395ba service neutron] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:41:11.433728 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e373f-28f9-40e9-aa76-2340ec2395ba service neutron] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:41:11.434001 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e373f-28f9-40e9-aa76-2340ec2395ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:41:11.435655 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-c82e373f-28f9-40e9-aa76-2340ec2395ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:41:11.549499 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e373f-28f9-40e9-aa76-2340ec2395ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:41:11.550904 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e373f-28f9-40e9-aa76-2340ec2395ba service neutron] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:41:11.569821 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 740/1470] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:41:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:41:20.962098 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=22 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:41:21.566999 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=66 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:41:31.936193 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c31aa-13c3-4268-980f-da9474b7e27d None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:41:31.936808 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c31aa-13c3-4268-980f-da9474b7e27d None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:41:31.936808 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c31aa-13c3-4268-980f-da9474b7e27d None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:41:31.938198 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 731/1471] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:41:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:41:31.944361 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd9e09-f76f-49b3-8629-45dd590d7657 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:41:31.944831 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd9e09-f76f-49b3-8629-45dd590d7657 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:41:31.944934 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd9e09-f76f-49b3-8629-45dd590d7657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:41:32.019494 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-6ffd9e09-f76f-49b3-8629-45dd590d7657 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:41:32.020747 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-6ffd9e09-f76f-49b3-8629-45dd590d7657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:41:32.131439 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 741/1472] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:41:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:41:32.434180 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05d4bb-a0a2-4f13-887b-aa6c416b2cf4 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:41:32.434472 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05d4bb-a0a2-4f13-887b-aa6c416b2cf4 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:41:32.434752 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05d4bb-a0a2-4f13-887b-aa6c416b2cf4 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:41:32.436281 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 732/1473] 10.222.0.20 () {54 vars in 962 bytes} [Thu Sep 22 02:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:41:32.441539 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613d24d-3ee3-401d-a899-0d088006ad1b None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:41:32.441830 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613d24d-3ee3-401d-a899-0d088006ad1b None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:41:32.442109 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613d24d-3ee3-401d-a899-0d088006ad1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:41:32.527057 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-9613d24d-3ee3-401d-a899-0d088006ad1b None None] MFA Rules not processed for user `6160520562b74be3b25c49a30e990d8b`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:41:32.528297 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-9613d24d-3ee3-401d-a899-0d088006ad1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:41:32.555818 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:41:32.620263 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 742/1474] 10.222.0.20 () {58 vars in 1059 bytes} [Thu Sep 22 02:41:32 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 02:41:32.627926 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb05d4bb-a0a2-4f13-887b-aa6c416b2cf4 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:41:32.629475 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-fb05d4bb-a0a2-4f13-887b-aa6c416b2cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:41:32.739908 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb05d4bb-a0a2-4f13-887b-aa6c416b2cf4 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:41:32.816131 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bf0931-4513-41db-a572-74ebe078faf9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160520562b74be3b25c49a30e990d8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:41:32.816738 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf0931-4513-41db-a572-74ebe078faf9 service glance] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:41:32.817017 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf0931-4513-41db-a572-74ebe078faf9 service glance] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:41:32.817285 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf0931-4513-41db-a572-74ebe078faf9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:41:32.818954 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-a8bf0931-4513-41db-a572-74ebe078faf9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:41:32.873510 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:41:32.936792 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bf0931-4513-41db-a572-74ebe078faf9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:41:32.938237 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bf0931-4513-41db-a572-74ebe078faf9 service glance] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:41:32.958009 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 733/1475] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:41:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 02:41:35.207033 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b57ca-edd2-4e5c-9a2c-1d12cd294975 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:41:35.207594 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b57ca-edd2-4e5c-9a2c-1d12cd294975 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:41:35.207594 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b57ca-edd2-4e5c-9a2c-1d12cd294975 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:41:35.209600 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 743/1476] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:41:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:41:35.219696 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c477c9b-795d-4e17-aa66-4c42c8a0f79d None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:41:35.220224 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c477c9b-795d-4e17-aa66-4c42c8a0f79d None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:41:35.220224 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c477c9b-795d-4e17-aa66-4c42c8a0f79d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:41:35.296572 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-3c477c9b-795d-4e17-aa66-4c42c8a0f79d None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:41:35.297711 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-3c477c9b-795d-4e17-aa66-4c42c8a0f79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:41:35.397721 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 734/1477] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:41:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 02:41:35.674218 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b57ca-edd2-4e5c-9a2c-1d12cd294975 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:41:35.675877 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-898b57ca-edd2-4e5c-9a2c-1d12cd294975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:41:35.823003 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b57ca-edd2-4e5c-9a2c-1d12cd294975 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:41:35.876598 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:41:35.922356 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b3a97-0dc9-4c03-b731-647132bb4d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f92fe1b089244bf4b21f4f98339aa81d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:41:35.923011 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b3a97-0dc9-4c03-b731-647132bb4d71 admin admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:41:35.923289 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b3a97-0dc9-4c03-b731-647132bb4d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:41:35.923556 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b3a97-0dc9-4c03-b731-647132bb4d71 admin admin] PATH_INFO: `/v3/projects` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:41:35.924196 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048b3a97-0dc9-4c03-b731-647132bb4d71 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:41:35.925997 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: WARNING py.warnings [None req-048b3a97-0dc9-4c03-b731-647132bb4d71 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 02:41:35.925997 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: warnings.warn(msg) Sep 22 02:41:35.925997 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: Sep 22 02:41:35.927593 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048b3a97-0dc9-4c03-b731-647132bb4d71 admin admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:41:35.947272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 744/1478] 10.222.0.20 () {56 vars in 1108 bytes} [Thu Sep 22 02:41:35 2022] GET /identity/v3/projects => generated 2553 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 02:41:38.132727 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52304ef-83a2-4fe7-ba90-efdc8e194e8d None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:41:38.133297 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52304ef-83a2-4fe7-ba90-efdc8e194e8d None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:41:38.133297 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52304ef-83a2-4fe7-ba90-efdc8e194e8d None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:41:38.134639 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 735/1479] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:41:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:41:38.140010 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810db46c-8b41-43a6-80a8-421db3fe98b3 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:41:38.140564 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810db46c-8b41-43a6-80a8-421db3fe98b3 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:41:38.140564 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810db46c-8b41-43a6-80a8-421db3fe98b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:41:38.207429 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-810db46c-8b41-43a6-80a8-421db3fe98b3 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:41:38.208666 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-810db46c-8b41-43a6-80a8-421db3fe98b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:41:38.307205 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 745/1480] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:41:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:41:38.602641 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52304ef-83a2-4fe7-ba90-efdc8e194e8d None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:41:38.604239 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-d52304ef-83a2-4fe7-ba90-efdc8e194e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:41:38.614158 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:41:38.756504 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52304ef-83a2-4fe7-ba90-efdc8e194e8d None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:41:38.849710 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:41:38.887245 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d45c3-ea01-4c10-9e66-ea62492ca66b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a64ed6c8e15d479887ab1dc59997d715', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:41:38.887868 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d45c3-ea01-4c10-9e66-ea62492ca66b service nova] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:41:38.888145 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d45c3-ea01-4c10-9e66-ea62492ca66b service nova] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:41:38.888413 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d45c3-ea01-4c10-9e66-ea62492ca66b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:41:38.890075 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-c41d45c3-ea01-4c10-9e66-ea62492ca66b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:41:38.984673 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41d45c3-ea01-4c10-9e66-ea62492ca66b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:41:38.986265 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41d45c3-ea01-4c10-9e66-ea62492ca66b service nova] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:41:39.005607 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 736/1481] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:41:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:41:41.377705 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87117ce-b459-4df3-8509-2f296f3217e6 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:41:41.378287 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87117ce-b459-4df3-8509-2f296f3217e6 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:41:41.378287 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87117ce-b459-4df3-8509-2f296f3217e6 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:41:41.379694 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 746/1482] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:41:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:41:41.385122 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfce7cb-c47c-467d-af28-3ad2d307d568 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:41:41.385570 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfce7cb-c47c-467d-af28-3ad2d307d568 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:41:41.385676 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfce7cb-c47c-467d-af28-3ad2d307d568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:41:41.454613 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-9bfce7cb-c47c-467d-af28-3ad2d307d568 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:41:41.455879 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-9bfce7cb-c47c-467d-af28-3ad2d307d568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:41:41.504292 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:41:41.558928 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 737/1483] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:41:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:41:41.854275 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87117ce-b459-4df3-8509-2f296f3217e6 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:41:41.855924 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-a87117ce-b459-4df3-8509-2f296f3217e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:41:41.880463 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:41:41.973086 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87117ce-b459-4df3-8509-2f296f3217e6 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:41:42.063953 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22cd662-44a0-4312-a0cb-c9c21cf75176 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f635200ad241b3b31ed49701bc511b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:41:42.064583 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22cd662-44a0-4312-a0cb-c9c21cf75176 service neutron] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:41:42.064868 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22cd662-44a0-4312-a0cb-c9c21cf75176 service neutron] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:41:42.065145 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22cd662-44a0-4312-a0cb-c9c21cf75176 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:41:42.066884 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-b22cd662-44a0-4312-a0cb-c9c21cf75176 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:41:42.154138 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:41:42.189788 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22cd662-44a0-4312-a0cb-c9c21cf75176 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:41:42.191272 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22cd662-44a0-4312-a0cb-c9c21cf75176 service neutron] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:41:42.211338 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 747/1484] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:41:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:41:44.510489 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a2cb0-f706-4b49-8146-977af48d0a49 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:41:44.511221 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a2cb0-f706-4b49-8146-977af48d0a49 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:41:44.511221 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a2cb0-f706-4b49-8146-977af48d0a49 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:41:44.512448 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 738/1485] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:41:44.518381 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e5f90-329f-41b7-9370-eb2bb256040b None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:41:44.518995 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e5f90-329f-41b7-9370-eb2bb256040b None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:41:44.518995 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e5f90-329f-41b7-9370-eb2bb256040b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:41:44.590988 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-081e5f90-329f-41b7-9370-eb2bb256040b None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:41:44.592412 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-081e5f90-329f-41b7-9370-eb2bb256040b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:41:44.693787 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 748/1486] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:41:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:41:44.985633 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a2cb0-f706-4b49-8146-977af48d0a49 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:41:44.987228 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-793a2cb0-f706-4b49-8146-977af48d0a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:41:45.099225 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a2cb0-f706-4b49-8146-977af48d0a49 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:41:45.176384 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:41:45.177940 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36738e4c-8ec4-481b-be22-38ecc4d99e5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f635200ad241b3b31ed49701bc511b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:41:45.178793 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36738e4c-8ec4-481b-be22-38ecc4d99e5f service neutron] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:41:45.179270 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36738e4c-8ec4-481b-be22-38ecc4d99e5f service neutron] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:41:45.179562 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36738e4c-8ec4-481b-be22-38ecc4d99e5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:41:45.181781 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-36738e4c-8ec4-481b-be22-38ecc4d99e5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:41:45.311758 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36738e4c-8ec4-481b-be22-38ecc4d99e5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:41:45.313167 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36738e4c-8ec4-481b-be22-38ecc4d99e5f service neutron] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:41:45.332269 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 739/1487] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:41:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:41:47.697640 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf992304-65a9-47ae-8ea3-481a37bcdead None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:41:47.698223 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf992304-65a9-47ae-8ea3-481a37bcdead None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:41:47.698223 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf992304-65a9-47ae-8ea3-481a37bcdead None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:41:47.699638 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 749/1488] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:41:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:41:47.705272 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08167dfa-a930-4ed6-84b4-c37ff6e42e87 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:41:47.705753 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08167dfa-a930-4ed6-84b4-c37ff6e42e87 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:41:47.705853 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08167dfa-a930-4ed6-84b4-c37ff6e42e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:41:47.774983 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-08167dfa-a930-4ed6-84b4-c37ff6e42e87 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:41:47.776403 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-08167dfa-a930-4ed6-84b4-c37ff6e42e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:41:47.830251 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:41:47.877282 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 740/1489] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:41:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:41:48.177381 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf992304-65a9-47ae-8ea3-481a37bcdead None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:41:48.178983 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-cf992304-65a9-47ae-8ea3-481a37bcdead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:41:48.220836 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:41:48.294704 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf992304-65a9-47ae-8ea3-481a37bcdead None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:41:48.372320 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea4ef1e-343d-4b4c-8fad-240ce057b73d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f635200ad241b3b31ed49701bc511b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:41:48.372942 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4ef1e-343d-4b4c-8fad-240ce057b73d service neutron] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:41:48.373232 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4ef1e-343d-4b4c-8fad-240ce057b73d service neutron] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:41:48.373509 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4ef1e-343d-4b4c-8fad-240ce057b73d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:41:48.375184 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-3ea4ef1e-343d-4b4c-8fad-240ce057b73d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:41:48.467527 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:41:48.500290 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea4ef1e-343d-4b4c-8fad-240ce057b73d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:41:48.501729 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea4ef1e-343d-4b4c-8fad-240ce057b73d service neutron] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:41:48.522729 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 750/1490] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:41:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:41:51.019686 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1491f-46f6-4b48-86bf-637b58c79172 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:41:51.020241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1491f-46f6-4b48-86bf-637b58c79172 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:41:51.020241 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1491f-46f6-4b48-86bf-637b58c79172 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:41:51.021659 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 741/1491] 10.222.0.20 () {54 vars in 927 bytes} [Thu Sep 22 02:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:41:51.027773 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a098f-27de-48ad-8972-aa74b83744ed None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:41:51.028234 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a098f-27de-48ad-8972-aa74b83744ed None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:41:51.028339 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a098f-27de-48ad-8972-aa74b83744ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:41:51.097664 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-7d7a098f-27de-48ad-8972-aa74b83744ed None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:41:51.099420 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-7d7a098f-27de-48ad-8972-aa74b83744ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:41:51.198085 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 751/1492] 10.222.0.20 () {58 vars in 1024 bytes} [Thu Sep 22 02:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:41:51.488459 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1491f-46f6-4b48-86bf-637b58c79172 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:41:51.490058 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-32b1491f-46f6-4b48-86bf-637b58c79172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:41:51.597743 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1491f-46f6-4b48-86bf-637b58c79172 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:41:51.673128 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660026ee-ba28-48a9-8cd9-4bd34063a2d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f635200ad241b3b31ed49701bc511b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:41:51.673749 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660026ee-ba28-48a9-8cd9-4bd34063a2d8 service neutron] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:41:51.674028 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660026ee-ba28-48a9-8cd9-4bd34063a2d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:41:51.674302 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660026ee-ba28-48a9-8cd9-4bd34063a2d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:41:51.675985 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-660026ee-ba28-48a9-8cd9-4bd34063a2d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:41:51.684777 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:41:51.789163 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660026ee-ba28-48a9-8cd9-4bd34063a2d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:41:51.790503 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660026ee-ba28-48a9-8cd9-4bd34063a2d8 service neutron] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:41:51.808861 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 742/1493] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:41:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:42:01.196400 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=82 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:42:01.806437 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=50 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:43:06.164838 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99316222-dfec-487e-9f31-88ef0a06f60e None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:43:06.165424 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99316222-dfec-487e-9f31-88ef0a06f60e None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:43:06.165424 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99316222-dfec-487e-9f31-88ef0a06f60e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:43:06.246642 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-99316222-dfec-487e-9f31-88ef0a06f60e None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:43:06.247892 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-99316222-dfec-487e-9f31-88ef0a06f60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:43:06.363326 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 752/1494] 10.222.0.20 () {58 vars in 958 bytes} [Thu Sep 22 02:43:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:43:06.372030 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9571d7-05c2-48dc-9d75-a9c37dd10ff7 None None] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:43:06.372611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9571d7-05c2-48dc-9d75-a9c37dd10ff7 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:43:06.372611 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9571d7-05c2-48dc-9d75-a9c37dd10ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:43:06.451431 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.auth.core [None req-cc9571d7-05c2-48dc-9d75-a9c37dd10ff7 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88065) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:43:06.452609 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-cc9571d7-05c2-48dc-9d75-a9c37dd10ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:43:06.561565 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 743/1495] 10.222.0.20 () {58 vars in 958 bytes} [Thu Sep 22 02:43:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:43:06.566549 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99316222-dfec-487e-9f31-88ef0a06f60e None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:43:06.567798 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-99316222-dfec-487e-9f31-88ef0a06f60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:43:06.579475 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99316222-dfec-487e-9f31-88ef0a06f60e None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:43:06.643632 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:43:06.681109 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456bd932-6846-4588-91f0-25cf9c954f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f92fe1b089244bf4b21f4f98339aa81d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:43:06.681721 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456bd932-6846-4588-91f0-25cf9c954f72 admin admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:43:06.682003 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456bd932-6846-4588-91f0-25cf9c954f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:43:06.682276 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456bd932-6846-4588-91f0-25cf9c954f72 admin admin] PATH_INFO: `/v3/domains` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:43:06.682959 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456bd932-6846-4588-91f0-25cf9c954f72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88064) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 02:43:06.683264 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456bd932-6846-4588-91f0-25cf9c954f72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:43:06.684279 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: WARNING py.warnings [None req-456bd932-6846-4588-91f0-25cf9c954f72 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 22 02:43:06.684279 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: warnings.warn(msg) Sep 22 02:43:06.684279 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: Sep 22 02:43:06.685171 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456bd932-6846-4588-91f0-25cf9c954f72 admin admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:43:06.696613 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 753/1496] 10.222.0.20 () {58 vars in 1153 bytes} [Thu Sep 22 02:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 02:43:06.705246 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9571d7-05c2-48dc-9d75-a9c37dd10ff7 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:43:06.715229 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9571d7-05c2-48dc-9d75-a9c37dd10ff7 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:43:06.777937 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:43:06.816502 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6dc38-9aad-420f-95aa-6de7f3aabf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f92fe1b089244bf4b21f4f98339aa81d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:43:06.817693 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6dc38-9aad-420f-95aa-6de7f3aabf74 admin admin] REQUEST_METHOD: `POST` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:43:06.817977 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6dc38-9aad-420f-95aa-6de7f3aabf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:43:06.818250 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6dc38-9aad-420f-95aa-6de7f3aabf74 admin admin] PATH_INFO: `/v3/projects` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:43:06.818924 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb6dc38-9aad-420f-95aa-6de7f3aabf74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:43:06.820753 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: WARNING py.warnings [None req-aeb6dc38-9aad-420f-95aa-6de7f3aabf74 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 02:43:06.820753 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: warnings.warn(msg) Sep 22 02:43:06.820753 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: Sep 22 02:43:06.821963 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb6dc38-9aad-420f-95aa-6de7f3aabf74 admin admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:43:06.883055 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 744/1497] 10.222.0.20 () {60 vars in 1153 bytes} [Thu Sep 22 02:43:06 2022] POST /identity/v3/projects => generated 377 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 02:43:06.887646 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456bd932-6846-4588-91f0-25cf9c954f72 admin admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:43:06.889164 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-456bd932-6846-4588-91f0-25cf9c954f72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:43:06.925410 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456bd932-6846-4588-91f0-25cf9c954f72 admin admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:43:07.024112 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99070729-0c6d-4162-8dc4-66d43c1056ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f92fe1b089244bf4b21f4f98339aa81d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:43:07.024658 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99070729-0c6d-4162-8dc4-66d43c1056ee admin admin] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:43:07.025503 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99070729-0c6d-4162-8dc4-66d43c1056ee admin admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:43:07.025503 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99070729-0c6d-4162-8dc4-66d43c1056ee admin admin] PATH_INFO: `/v3/users` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:43:07.025836 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99070729-0c6d-4162-8dc4-66d43c1056ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:43:07.026881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: WARNING py.warnings [None req-99070729-0c6d-4162-8dc4-66d43c1056ee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 02:43:07.026881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: warnings.warn(msg) Sep 22 02:43:07.026881 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: Sep 22 02:43:07.028376 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99070729-0c6d-4162-8dc4-66d43c1056ee admin admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:43:07.072362 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 754/1498] 10.222.0.20 () {60 vars in 1144 bytes} [Thu Sep 22 02:43:06 2022] POST /identity/v3/users => generated 289 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 02:43:07.076573 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6dc38-9aad-420f-95aa-6de7f3aabf74 admin admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:43:07.078134 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-aeb6dc38-9aad-420f-95aa-6de7f3aabf74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:43:07.142209 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6dc38-9aad-420f-95aa-6de7f3aabf74 admin admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:43:07.228988 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:43:07.242914 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3e003-7938-4df1-ac64-d816a05c1080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f92fe1b089244bf4b21f4f98339aa81d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:43:07.243572 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3e003-7938-4df1-ac64-d816a05c1080 admin admin] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:43:07.243856 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3e003-7938-4df1-ac64-d816a05c1080 admin admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:43:07.244129 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3e003-7938-4df1-ac64-d816a05c1080 admin admin] PATH_INFO: `/v3/roles` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:43:07.244780 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a3e003-7938-4df1-ac64-d816a05c1080 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:43:07.245802 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: WARNING py.warnings [None req-82a3e003-7938-4df1-ac64-d816a05c1080 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 22 02:43:07.245802 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: warnings.warn(msg) Sep 22 02:43:07.245802 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: Sep 22 02:43:07.246715 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a3e003-7938-4df1-ac64-d816a05c1080 admin admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:43:07.260322 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 745/1499] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Sep 22 02:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 02:43:07.264381 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99070729-0c6d-4162-8dc4-66d43c1056ee admin admin] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:43:07.265925 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-99070729-0c6d-4162-8dc4-66d43c1056ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:43:07.277822 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99070729-0c6d-4162-8dc4-66d43c1056ee admin admin] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:43:07.304743 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:43:07.382530 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36b724-9ac7-4f07-a614-59ba3c54e59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f92fe1b089244bf4b21f4f98339aa81d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:43:07.383175 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36b724-9ac7-4f07-a614-59ba3c54e59a admin admin] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:43:07.383460 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36b724-9ac7-4f07-a614-59ba3c54e59a admin admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:43:07.383736 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36b724-9ac7-4f07-a614-59ba3c54e59a admin admin] PATH_INFO: `/v3/roles` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:43:07.384391 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36b724-9ac7-4f07-a614-59ba3c54e59a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:43:07.385416 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: WARNING py.warnings [None req-ad36b724-9ac7-4f07-a614-59ba3c54e59a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 22 02:43:07.385416 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: warnings.warn(msg) Sep 22 02:43:07.385416 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: Sep 22 02:43:07.386314 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36b724-9ac7-4f07-a614-59ba3c54e59a admin admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:43:07.399462 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 755/1500] 10.222.0.20 () {58 vars in 1122 bytes} [Thu Sep 22 02:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 02:43:07.403969 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3e003-7938-4df1-ac64-d816a05c1080 admin admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:43:07.405501 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-82a3e003-7938-4df1-ac64-d816a05c1080 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:43:07.440311 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3e003-7938-4df1-ac64-d816a05c1080 admin admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:43:07.559683 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7388f41-17f1-4080-a8ff-b1b096e47e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f92fe1b089244bf4b21f4f98339aa81d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:43:07.560355 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7388f41-17f1-4080-a8ff-b1b096e47e37 admin admin] REQUEST_METHOD: `PUT` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:43:07.560638 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7388f41-17f1-4080-a8ff-b1b096e47e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:43:07.560910 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7388f41-17f1-4080-a8ff-b1b096e47e37 admin admin] PATH_INFO: `/v3/projects/5bbbb85050bf434ba97861ccca29496c/users/72f5437d02914f7eb2d04608516c845c/roles/83efac3b92eb44e783f4924c9d9ee842` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:43:07.585004 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7388f41-17f1-4080-a8ff-b1b096e47e37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bbbb85050bf434ba97861ccca29496c, user_id=72f5437d02914f7eb2d04608516c845c, role_id=83efac3b92eb44e783f4924c9d9ee842)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:43:07.586059 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: WARNING py.warnings [None req-d7388f41-17f1-4080-a8ff-b1b096e47e37 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 02:43:07.586059 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: warnings.warn(msg) Sep 22 02:43:07.586059 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: Sep 22 02:43:07.588064 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7388f41-17f1-4080-a8ff-b1b096e47e37 admin admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:43:07.617398 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 746/1501] 10.222.0.20 () {60 vars in 1483 bytes} [Thu Sep 22 02:43:07 2022] PUT /identity/v3/projects/5bbbb85050bf434ba97861ccca29496c/users/72f5437d02914f7eb2d04608516c845c/roles/83efac3b92eb44e783f4924c9d9ee842 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 02:43:07.629643 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096f1ba-d6a5-47d7-a7e7-526d59f6e1fc None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:43:07.629933 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096f1ba-d6a5-47d7-a7e7-526d59f6e1fc None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:43:07.630217 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096f1ba-d6a5-47d7-a7e7-526d59f6e1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:43:07.692847 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-4096f1ba-d6a5-47d7-a7e7-526d59f6e1fc None None] MFA Rules not processed for user `72f5437d02914f7eb2d04608516c845c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:43:07.694110 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-4096f1ba-d6a5-47d7-a7e7-526d59f6e1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:43:07.774093 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 756/1502] 10.222.0.20 () {58 vars in 958 bytes} [Thu Sep 22 02:43:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:43:07.788422 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37529ec9-820b-46ca-81cd-2dee031e0fe1 None None] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:43:07.788713 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37529ec9-820b-46ca-81cd-2dee031e0fe1 None None] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:43:07.788990 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37529ec9-820b-46ca-81cd-2dee031e0fe1 None None] PATH_INFO: `/` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:43:07.790350 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 747/1503] 10.222.0.20 () {54 vars in 963 bytes} [Thu Sep 22 02:43:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:43:07.795355 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a379df1-4b70-4163-bd7a-c71c9ce34dcb None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:43:07.795641 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a379df1-4b70-4163-bd7a-c71c9ce34dcb None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:43:07.795938 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a379df1-4b70-4163-bd7a-c71c9ce34dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:43:07.839314 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:43:07.898844 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-8a379df1-4b70-4163-bd7a-c71c9ce34dcb None None] MFA Rules not processed for user `a08a0ed98bc14328a3d24309d7996b51`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:43:07.900104 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-8a379df1-4b70-4163-bd7a-c71c9ce34dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:43:07.981046 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 757/1504] 10.222.0.20 () {58 vars in 1060 bytes} [Thu Sep 22 02:43:07 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:43:07.991186 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37529ec9-820b-46ca-81cd-2dee031e0fe1 None None] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:43:07.992755 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-37529ec9-820b-46ca-81cd-2dee031e0fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:43:08.046188 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:43:08.105559 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37529ec9-820b-46ca-81cd-2dee031e0fe1 None None] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:43:08.187369 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10cd720-0c0b-4ec4-8887-b781ced4c45a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08a0ed98bc14328a3d24309d7996b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:43:08.187988 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10cd720-0c0b-4ec4-8887-b781ced4c45a service cinder] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:43:08.188275 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10cd720-0c0b-4ec4-8887-b781ced4c45a service cinder] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:43:08.188554 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10cd720-0c0b-4ec4-8887-b781ced4c45a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:43:08.190202 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-b10cd720-0c0b-4ec4-8887-b781ced4c45a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:43:08.296543 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10cd720-0c0b-4ec4-8887-b781ced4c45a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:43:08.297912 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10cd720-0c0b-4ec4-8887-b781ced4c45a service cinder] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:43:08.301633 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:43:08.317651 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 748/1505] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:43:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:43:08.364798 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18abe791-efae-45ef-999b-3284e21d08d6 None None] REQUEST_METHOD: `GET` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:43:08.365254 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18abe791-efae-45ef-999b-3284e21d08d6 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:43:08.365373 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18abe791-efae-45ef-999b-3284e21d08d6 None None] PATH_INFO: `/` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:43:08.366657 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 758/1506] 10.222.0.20 () {52 vars in 830 bytes} [Thu Sep 22 02:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 02:43:08.370363 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10cd720-0c0b-4ec4-8887-b781ced4c45a service cinder] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:43:08.371629 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-b10cd720-0c0b-4ec4-8887-b781ced4c45a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:43:08.482599 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10cd720-0c0b-4ec4-8887-b781ced4c45a service cinder] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:43:08.587023 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5bb79-67b2-40ff-b7f0-ed65174007ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f92fe1b089244bf4b21f4f98339aa81d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:43:08.587746 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5bb79-67b2-40ff-b7f0-ed65174007ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:43:08.587949 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5bb79-67b2-40ff-b7f0-ed65174007ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:43:08.588218 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5bb79-67b2-40ff-b7f0-ed65174007ba admin admin] PATH_INFO: `/v3/users/72f5437d02914f7eb2d04608516c845c` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:43:08.602469 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd5bb79-67b2-40ff-b7f0-ed65174007ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=72f5437d02914f7eb2d04608516c845c)` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:43:08.603556 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: WARNING py.warnings [None req-cdd5bb79-67b2-40ff-b7f0-ed65174007ba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 02:43:08.603556 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: warnings.warn(msg) Sep 22 02:43:08.603556 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: Sep 22 02:43:08.604864 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd5bb79-67b2-40ff-b7f0-ed65174007ba admin admin] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:43:08.616477 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=100 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:43:08.681883 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.notifications [None req-cdd5bb79-67b2-40ff-b7f0-ed65174007ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72f5437d02914f7eb2d04608516c845c'} {{(pid=88065) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 02:43:08.707354 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.notifications [None req-cdd5bb79-67b2-40ff-b7f0-ed65174007ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72f5437d02914f7eb2d04608516c845c'} {{(pid=88065) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 02:43:08.707885 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.notifications [None req-cdd5bb79-67b2-40ff-b7f0-ed65174007ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72f5437d02914f7eb2d04608516c845c'} {{(pid=88065) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 02:43:08.723465 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.notifications [None req-cdd5bb79-67b2-40ff-b7f0-ed65174007ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72f5437d02914f7eb2d04608516c845c'} {{(pid=88065) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 02:43:08.733569 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.notifications [None req-cdd5bb79-67b2-40ff-b7f0-ed65174007ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72f5437d02914f7eb2d04608516c845c'} {{(pid=88065) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 02:43:08.740511 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 749/1507] 10.222.0.20 () {58 vars in 1224 bytes} [Thu Sep 22 02:43:08 2022] DELETE /identity/v3/users/72f5437d02914f7eb2d04608516c845c => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 02:43:08.745128 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc69f3-815c-4f2c-9765-0933f2e9c0d1 None None] REQUEST_METHOD: `POST` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:43:08.745479 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc69f3-815c-4f2c-9765-0933f2e9c0d1 None None] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:43:08.745822 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc69f3-815c-4f2c-9765-0933f2e9c0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:43:08.805644 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.auth.core [None req-ffcc69f3-815c-4f2c-9765-0933f2e9c0d1 None None] MFA Rules not processed for user `778bd65251774785bdd2460a6778044c`. Rule list: `[]` (Enabled: `True`). {{(pid=88064) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 02:43:08.806920 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-ffcc69f3-815c-4f2c-9765-0933f2e9c0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:43:08.885501 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=100 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:43:08.898467 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 759/1508] 10.222.0.20 () {58 vars in 958 bytes} [Thu Sep 22 02:43:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 02:43:08.908454 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5bb79-67b2-40ff-b7f0-ed65174007ba admin admin] Authenticating user token {{(pid=88065) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:43:08.909705 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-cdd5bb79-67b2-40ff-b7f0-ed65174007ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:43:09.052721 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5bb79-67b2-40ff-b7f0-ed65174007ba admin admin] Validating token access rules against request {{(pid=88065) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:43:09.108412 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:43:09.130203 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9639700-1654-4a14-94a8-f0612398b522 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f635200ad241b3b31ed49701bc511b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a792f8dc45145d687ddd58c94d44b5e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88065) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:43:09.130852 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9639700-1654-4a14-94a8-f0612398b522 service neutron] REQUEST_METHOD: `GET` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:43:09.131135 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9639700-1654-4a14-94a8-f0612398b522 service neutron] SCRIPT_NAME: `/identity` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:43:09.131414 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9639700-1654-4a14-94a8-f0612398b522 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88065) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:43:09.133027 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.fernet_utils [None req-f9639700-1654-4a14-94a8-f0612398b522 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88065) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:43:09.244102 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9639700-1654-4a14-94a8-f0612398b522 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:43:09.245461 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9639700-1654-4a14-94a8-f0612398b522 service neutron] RBAC: Authorization granted {{(pid=88065) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:43:09.263407 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: [pid: 88065|app: 0|req: 750/1509] 10.222.0.20 () {60 vars in 1365 bytes} [Thu Sep 22 02:43:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 02:43:09.623429 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcc69f3-815c-4f2c-9765-0933f2e9c0d1 None None] Authenticating user token {{(pid=88064) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 02:43:09.624889 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.fernet_utils [None req-ffcc69f3-815c-4f2c-9765-0933f2e9c0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88064) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 02:43:09.636690 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcc69f3-815c-4f2c-9765-0933f2e9c0d1 None None] Validating token access rules against request {{(pid=88064) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 02:43:09.748910 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d53ae3-5b8e-4d80-ad32-dc0efc7b42ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '778bd65251774785bdd2460a6778044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f92fe1b089244bf4b21f4f98339aa81d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88064) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 02:43:09.749549 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d53ae3-5b8e-4d80-ad32-dc0efc7b42ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 02:43:09.749825 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d53ae3-5b8e-4d80-ad32-dc0efc7b42ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 02:43:09.750098 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d53ae3-5b8e-4d80-ad32-dc0efc7b42ad admin admin] PATH_INFO: `/v3/projects/5bbbb85050bf434ba97861ccca29496c` {{(pid=88064) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 02:43:09.761797 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d53ae3-5b8e-4d80-ad32-dc0efc7b42ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bbbb85050bf434ba97861ccca29496c)` {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 02:43:09.762884 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: WARNING py.warnings [None req-87d53ae3-5b8e-4d80-ad32-dc0efc7b42ad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 02:43:09.762884 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: warnings.warn(msg) Sep 22 02:43:09.762884 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: Sep 22 02:43:09.764150 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d53ae3-5b8e-4d80-ad32-dc0efc7b42ad admin admin] RBAC: Authorization granted {{(pid=88064) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 02:43:09.829076 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.notifications [None req-87d53ae3-5b8e-4d80-ad32-dc0efc7b42ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bbbb85050bf434ba97861ccca29496c', 'request_id': 'req-87d53ae3-5b8e-4d80-ad32-dc0efc7b42ad'} {{(pid=88064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 02:43:09.840314 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.notifications [None req-87d53ae3-5b8e-4d80-ad32-dc0efc7b42ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bbbb85050bf434ba97861ccca29496c', 'request_id': 'req-87d53ae3-5b8e-4d80-ad32-dc0efc7b42ad'} {{(pid=88064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 02:43:09.857884 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.notifications [None req-87d53ae3-5b8e-4d80-ad32-dc0efc7b42ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bbbb85050bf434ba97861ccca29496c', 'request_id': 'req-87d53ae3-5b8e-4d80-ad32-dc0efc7b42ad'} {{(pid=88064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 02:43:09.884584 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.notifications [None req-87d53ae3-5b8e-4d80-ad32-dc0efc7b42ad admin admin] The token cache is being invalidate because project 5bbbb85050bf434ba97861ccca29496c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88064) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 02:43:09.884882 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.notifications [None req-87d53ae3-5b8e-4d80-ad32-dc0efc7b42ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 02:43:09.885430 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG keystone.notifications [None req-87d53ae3-5b8e-4d80-ad32-dc0efc7b42ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88064) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 02:43:09.886767 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: [pid: 88064|app: 0|req: 760/1510] 10.222.0.20 () {58 vars in 1233 bytes} [Thu Sep 22 02:43:09 2022] DELETE /identity/v3/projects/5bbbb85050bf434ba97861ccca29496c => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 02:43:19.261110 dv-cinder-857075-7-iscsi devstack@keystone.service[88065]: DEBUG dbcounter [-] [88065] Writing DB stats keystone:SELECT=62 {{(pid=88065) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 02:43:19.882374 dv-cinder-857075-7-iscsi devstack@keystone.service[88064]: DEBUG dbcounter [-] [88064] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88064) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}